OpenRaiser/Intern-Atlas huggingface dataset:
ConSTR: A Contextual Search Term Recommender
Venue: arXiv cs.DL 2021
Year: 2021

In this demo paper, we present ConSTR, a novel Contextual Search Term Recommender that utilises the user's interaction context for search term recommendation and literature retrieval. ConSTR integrates a two-layered recommendation interface: the first layer suggests terms with respect to a user's current search term, and the second layer suggests terms based on the users' previous search activities (interaction context). For the demonstration, ConSTR is built on the arXiv, an academic repository consisting of 1.8 million documents.
Adaptive Machine Unlearning
Venue: arXiv cs.LG 2021
Year: 2021

Data deletion algorithms aim to remove the influence of deleted data points from trained models at a cheaper computational cost than fully retraining those models. However, for sequences of deletions, most prior work in the non-convex setting gives valid guarantees only for sequences that are chosen independently of the models that are published. If people choose to delete their data as a function of the published models (because they don't like what the models reveal about them, for example), then the update sequence is adaptive. In this paper, we give a general reduction from deletion guarantees against adaptive sequences to deletion guarantees against non-adaptive sequences, using differential privacy and its connection to max information. Combined with ideas from prior work which give guarantees for non-adaptive deletion sequences, this leads to extremely flexible algorithms able to handle arbitrary model classes and training methodologies, giving strong provable deletion guarantees for adaptive deletion sequences. We show in theory how prior work for non-convex models fails against adaptive deletion sequences, and use this intuition to design a practical attack against the SISA algorithm of Bourtoule et al. [2021] on CIFAR-10, MNIST, Fashion-MNIST.
Lifts for Voronoi cells of lattices
Venue: arXiv cs.DM 2021
Year: 2021

Many polytopes arising in polyhedral combinatorics are linear projections of higher-dimensional polytopes with significantly fewer facets. Such lifts may yield compressed representations of polytopes, which are typically used to construct small-size linear programs. Motivated by algorithmic implications for the closest vector problem, we study lifts of Voronoi cells of lattices. We construct an explicit $d$-dimensional lattice such that every lift of the respective Voronoi cell has $2^{Ω(d / \log d)}$ facets. On the positive side, we show that Voronoi cells of $d$-dimensional root lattices and their dual lattices have lifts with $O(d)$ and $O(d \log d)$ facets, respectively. We obtain similar results for spectrahedral lifts.
Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation
Venue: arXiv cs.CL 2021
Year: 2021

Answering a programming question using only its title is difficult as salient contextual information is omitted. Based on this observation, we present a corpus of over 40,000 StackOverflow question texts to be used in conjunction with their corresponding intents from the CoNaLa dataset (Yin et al., 2018). Using both the intent and question body, we use BART to establish a baseline BLEU score of 34.35 for this new task. We find further improvements of $2.8\%$ by combining the mined CoNaLa data with the labeled data to achieve a 35.32 BLEU score. We evaluate prior state-of-the-art CoNaLa models with this additional data and find that our proposed method of using the body and mined data beats the BLEU score of the prior state-of-the-art by $71.96\%$. Finally, we perform ablations to demonstrate that BART is an unsupervised multimodal learner and examine its extractive behavior. The code and data can be found https://github.com/gabeorlanski/stackoverflow-encourages-cheating.
Muddling Label Regularization: Deep Learning for Tabular Datasets
Venue: arXiv cs.LG 2021
Year: 2021

Deep Learning (DL) is considered the state-of-the-art in computer vision, speech recognition and natural language processing. Until recently, it was also widely accepted that DL is irrelevant for learning tasks on tabular data, especially in the small sample regime where ensemble methods are acknowledged as the gold standard. We present a new end-to-end differentiable method to train a standard FFNN. Our method, \textbf{Muddling labels for Regularization} (\texttt{MLR}), penalizes memorization through the generation of uninformative labels and the application of a differentiable close-form regularization scheme on the last hidden layer during training. \texttt{MLR} outperforms classical NN and the gold standard (GBDT, RF) for regression and classification tasks on several datasets from the UCI database and Kaggle covering a large range of sample sizes and feature to sample ratios. Researchers and practitioners can use \texttt{MLR} on its own as an off-the-shelf \DL{} solution or integrate it into the most advanced ML pipelines.
Private Multi-Group Aggregation
Venue: arXiv cs.IT 2021
Year: 2021

We study the differentially private multi group aggregation (PMGA) problem. This setting involves a single server and $n$ users. Each user belongs to one of $k$ distinct groups and holds a discrete value. The goal is to design schemes that allow the server to find the aggregate (sum) of the values in each group (with high accuracy) under communication and local differential privacy constraints. The privacy constraint guarantees that the user's group remains private. This is motivated by applications where a user's group can reveal sensitive information, such as his religious and political beliefs, health condition, or race. We propose a novel scheme, dubbed Query and Aggregate (Q\&A) for PMGA. The novelty of Q\&A is that it is an interactive aggregation scheme. In Q\&A, each user is assigned a random query matrix, to which he sends the server an answer based on his group and value. We characterize the Q\&A scheme's performance in terms of accuracy (MSE), privacy, and communication. We compare Q\&A to the Randomized Group (RG) scheme, which is non-interactive and adapts existing randomized response schemes to the PMGA setting. We observe that typically Q\&A outperforms RG, in terms of privacy vs. utility, in the high privacy regime.
Assessing Open Interfaces and Protocols of PLCs for Computation Offloading at Field Level
Venue: arXiv cs.DC 2021
Year: 2021

Programmable logic controllers (PLCs) are the core element of industrial plants in todays deployments. They read sensor values, execute control algorithms, and write output values. Furthermore, industrial plants have lifetimes of one or more decades. Thus, in a realistic Industry 4.0 scenario, these devices have to be integrated in novel systems. In order to apply advanced concepts and technologies, such as computation offloading, which requires data exchange between PLCs and edge cloud, we investigate open communication interfaces of two typical PLCs of Siemens S7 series. Hence, each of the interfaces is analyzed based on plug & play capability, if metadata is provided, protocol efficiency, and performance. For the latter, the smallest possible update time for each of the interfaces will be measured.
Automatic Generation of Machine Learning Synthetic Data Using ROS
Venue: arXiv cs.LG 2021
Year: 2021

Data labeling is a time intensive process. As such, many data scientists use various tools to aid in the data generation and labeling process. While these tools help automate labeling, many still require user interaction throughout the process. Additionally, most target only a few network frameworks. Any researchers exploring multiple frameworks must find additional tools orwrite conversion scripts. This paper presents an automated tool for generating synthetic data in arbitrary network formats. It uses Robot Operating System (ROS) and Gazebo, which are common tools in the robotics community. Through ROS paradigms, it allows extensive user customization of the simulation environment and data generation process. Additionally, a plugin-like framework allows the development of arbitrary data format writers without the need to change the main body of code. Using this tool, the authors were able to generate an arbitrarily large image dataset for three unique training formats using approximately 15 min of user setup time and a variable amount of hands-off run time, depending on the dataset size. The source code for this data generation tool is available at https://github.com/Navy-RISE-Lab/nn_data_collection
DNS attack mitigation Using OpenStack Isolation
Venue: arXiv cs.CR 2021
Year: 2021

The Domain Name System (DNS) is essential for the Internet, giving a mechanism to resolve hostnames into Internet Protocol (IP) addresses. DNS is known as the world's largest distributed database that manages hostnames and Internet Protocol. By having the DNS, only simple names that can be easily memorized will be used and then the domain name system will map it into the numeric Internet Protocol addresses that are used by computers to communicate. This research aims to propose a model for the development of a private cloud infrastructure to host DNS. The cloud infrastructure will be created using the OpenStack software platform where each server will be hosted separately in a different virtual machine. Virtual network architecture will be created using the Software Defined Networking (SDN) approach and it will be secured using Firewall as a Service (FWaaS). By hosting DNS in private cloud infrastructure, the DNS servers will be out of reach by attackers which will prevent DNS attacks. Besides, available research had proven that the cloud is the best choice for DNS. A prototype had been implemented and evaluated for its efficiencies. The findings from the evaluation carried out shown a positive result.
Neural Extractive Search
Venue: arXiv cs.CL 2021
Year: 2021

Domain experts often need to extract structured information from large corpora. We advocate for a search paradigm called ``extractive search'', in which a search query is enriched with capture-slots, to allow for such rapid extraction. Such an extractive search system can be built around syntactic structures, resulting in high-precision, low-recall results. We show how the recall can be improved using neural retrieval and alignment. The goals of this paper are to concisely introduce the extractive-search paradigm; and to demonstrate a prototype neural retrieval system for extractive search and its benefits and potential. Our prototype is available at \url{https://spike.neural-sim.apps.allenai.org/} and a video demonstration is available at \url{https://vimeo.com/559586687}.
Don't Get Yourself into Trouble! Risk-aware Decision-Making for Autonomous Vehicles
Venue: arXiv cs.AI 2021
Year: 2021

Risk is traditionally described as the expected likelihood of an undesirable outcome, such as collisions for autonomous vehicles. Accurately predicting risk or potentially risky situations is critical for the safe operation of autonomous vehicles. In our previous work, we showed that risk could be characterized by two components: 1) the probability of an undesirable outcome and 2) an estimate of how undesirable the outcome is (loss). This paper is an extension to our previous work. In this paper, using our trained deep reinforcement learning model for navigating around crowds, we developed a risk-based decision-making framework for the autonomous vehicle that integrates the high-level risk-based path planning with the reinforcement learning-based low-level control. We evaluated our method in a high-fidelity simulation such as CARLA. This work can improve safety by allowing an autonomous vehicle to one day avoid and react to risky situations.
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
Venue: arXiv cs.CV 2021
Year: 2021

Most existing video-and-language (VidL) research focuses on a single dataset, or multiple datasets of a single task. In reality, a truly useful VidL system is expected to be easily generalizable to diverse tasks, domains, and datasets. To facilitate the evaluation of such systems, we introduce Video-And-Language Understanding Evaluation (VALUE) benchmark, an assemblage of 11 VidL datasets over 3 popular tasks: (i) text-to-video retrieval; (ii) video question answering; and (iii) video captioning. VALUE benchmark aims to cover a broad range of video genres, video lengths, data volumes, and task difficulty levels. Rather than focusing on single-channel videos with visual information only, VALUE promotes models that leverage information from both video frames and their associated subtitles, as well as models that share knowledge across multiple tasks. We evaluate various baseline methods with and without large-scale VidL pre-training, and systematically investigate the impact of video input channels, fusion methods, and different video representations. We also study the transferability between tasks, and conduct multi-task learning under different settings. The significant gap between our best model and human performance calls for future study for advanced VidL models. VALUE is available at https://value-benchmark.github.io/.
On the Explanation of Similarity for Developing and Deploying CBR Systems
Venue: arXiv cs.HC 2021
Year: 2021

During the early stages of developing Case-Based Reasoning (CBR) systems the definition of similarity measures is challenging since this task requires transferring implicit knowledge of domain experts into knowledge representations. While an entire CBR system is very explanatory, the similarity measure determines the ranking but do not necessarily show which features contribute to high (or low) rankings. In this paper we present our work on opening the knowledge engineering process for similarity modelling. This work present is a result of an interdisciplinary research collaboration between AI and public health researchers developing e-Health applications. During this work explainability and transparency of the development process is crucial to allow in-depth quality assurance of the by the domain experts.
Incentivizing Efficient Equilibria in Traffic Networks with Mixed Autonomy
Venue: arXiv cs.MA 2021
Year: 2021

Traffic congestion has large economic and social costs. The introduction of autonomous vehicles can potentially reduce this congestion by increasing road capacity via vehicle platooning and by creating an avenue for influencing people's choice of routes. We consider a network of parallel roads with two modes of transportation: (i) human drivers, who will choose the quickest route available to them, and (ii) a ride hailing service, which provides an array of autonomous vehicle route options, each with different prices, to users. We formalize a model of vehicle flow in mixed autonomy and a model of how autonomous service users make choices between routes with different prices and latencies. Developing an algorithm to learn the preferences of the users, we formulate a planning optimization that chooses prices to maximize a social objective. We demonstrate the benefit of the proposed scheme by comparing the results to theoretical benchmarks which we show can be efficiently calculated.
A tipping point for open citation data
Venue: arXiv cs.DL 2021
Year: 2021

Open citation data can improve the transparency and robustness of scientific portfolio analysis, improve science policy decision-making, stimulate downstream commercial activity, and increase the discoverability of scientific articles. Once sparsely populated, public-domain citation databases crossed a threshold of one billion citations in February 2021, during the COVID-19 pandemic. Shortly thereafter, the threshold of one billion public-domain citations from the Crossref database alone. Since the relative advantage of withholding data in closed databases has diminished with the flood of public-domain data, this likely constitutes an irreversible change in the citation data ecosystem. The successes of this movement can guide future open data efforts.
Boolean Matrix Factorization via Nonnegative Auxiliary Optimization
Venue: arXiv cs.DS 2021
Year: 2021

A novel approach to Boolean matrix factorization (BMF) is presented. Instead of solving the BMF problem directly, this approach solves a nonnegative optimization problem with the constraint over an auxiliary matrix whose Boolean structure is identical to the initial Boolean data. Then the solution of the nonnegative auxiliary optimization problem is thresholded to provide a solution for the BMF problem. We provide the proofs for the equivalencies of the two solution spaces under the existence of an exact solution. Moreover, the nonincreasing property of the algorithm is also proven. Experiments on synthetic and real datasets are conducted to show the effectiveness and complexity of the algorithm compared to other current methods.
Analysis of Attacker Behavior in Compromised Hosts During Command and Control
Venue: arXiv cs.CR 2021
Year: 2021

Traditional reactive approach of blacklisting botnets fails to adapt to the rapidly evolving landscape of cyberattacks. An automated and proactive approach to detect and block botnet hosts will immensely benefit the industry. Behavioral analysis of botnet is shown to be effective against a wide variety of attack types. Current works, however, focus solely on analyzing network traffic from and to the bots. In this work we take a different approach of analyzing the chain of commands input by attackers in a compromised host. We have deployed several honeypots to simulate Linux shells and allowed attackers access to the shells to collect a large dataset of commands. We have further developed an automated mechanism to analyze these data. For the automation we have developed a system called CYbersecurity information Exchange with Privacy (CYBEX-P). Finally, we have done a sequential analysis on the dataset to show that we can successfully predict attacker behavior from the shell commands without analyzing network traffic like previous works.
OODIn: An Optimised On-Device Inference Framework for Heterogeneous Mobile Devices
Venue: arXiv cs.LG 2021
Year: 2021

Radical progress in the field of deep learning (DL) has led to unprecedented accuracy in diverse inference tasks. As such, deploying DL models across mobile platforms is vital to enable the development and broad availability of the next-generation intelligent apps. Nevertheless, the wide and optimised deployment of DL models is currently hindered by the vast system heterogeneity of mobile devices, the varying computational cost of different DL models and the variability of performance needs across DL applications. This paper proposes OODIn, a framework for the optimised deployment of DL apps across heterogeneous mobile devices. OODIn comprises a novel DL-specific software architecture together with an analytical framework for modelling DL applications that: (1) counteract the variability in device resources and DL models by means of a highly parametrised multi-layer design; and (2) perform a principled optimisation of both model- and system-level parameters through a multi-objective formulation, designed for DL inference apps, in order to adapt the deployment to the user-specified performance requirements and device capabilities. Quantitative evaluation shows that the proposed framework consistently outperforms status-quo designs across heterogeneous devices and delivers up to 4.3x and 3.5x performance gain over highly optimised platform- and model-aware designs respectively, while effectively adapting execution to dynamic changes in resource availability.
Validating Static Warnings via Testing Code Fragments
Venue: arXiv cs.SE 2021
Year: 2021

Static analysis is an important approach for finding bugs and vulnerabilities in software. However, inspecting and confirming static warnings are challenging and time-consuming. In this paper, we present a novel solution that automatically generates test cases based on static warnings to validate true and false positives. We designed a syntactic patching algorithm that can generate syntactically valid, semantic preserving executable code fragments from static warnings. We developed a build and testing system to automatically test code fragments using fuzzers, KLEE and Valgrind. We evaluated our techniques using 12 real-world C projects and 1955 warnings from two commercial static analysis tools. We successfully built 68.5% code fragments and generated 1003 test cases. Through automatic testing, we identified 48 true positives and 27 false positives, and 205 likely false positives. We matched 4 CVE and real-world bugs using Helium, and they are only triggered by our tool but not other baseline tools. We found that testing code fragments is scalable and useful; it can trigger bugs that testing entire programs or testing procedures failed to trigger.
Cooperative Beamforming for Wireless Fronthaul and Access Links in Ultra-Dense C-RANs with SWIPT: A First-Order Approach
Venue: arXiv cs.IT 2021
Year: 2021

This work studies multigroup multicasting transmission in cloud radio access networks (C-RANs) with simultaneous wireless information and power transfer, where densely packed remote radio heads (RRHs) cooperatively provide information and energy services for information users (IUs) and energy users (EUs), respectively. To maximize the weighted sum rate (WSR) of information services while satisfying the energy harvesting levels at EUs, an optimization of joint beamforming design for the fronthaul and access links is formulated, which is however neither smooth nor convex and is indeed NP-hard. To tackle this difficulty, the smooth and successive convex approximations are used to transform the original problem into a sequence of convex problems, and two first-order algorithms are developed to find the initial feasible point and the nearly optimal solution, respectively. Moreover, an accelerated algorithm is designed to improve the convergence speed by exploiting both Nesterov and heavy-ball momentums. Numerical results demonstrate that the proposed first-order algorithms achieve almost the same WSR as that of traditional second-order approaches yet with much lower computational complexity, and the proposed scheme outperforms state-of-the-art competing schemes in terms of WSR.
On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation
Venue: arXiv cs.CL 2021
Year: 2021

In the recent advances of natural language processing, the scale of the state-of-the-art models and datasets is usually extensive, which challenges the application of sample-based explanation methods in many aspects, such as explanation interpretability, efficiency, and faithfulness. In this work, for the first time, we can improve the interpretability of explanations by allowing arbitrary text sequences as the explanation unit. On top of this, we implement a hessian-free method with a model faithfulness guarantee. Finally, to compare our method with the others, we propose a semantic-based evaluation metric that can better align with humans' judgment of explanations than the widely adopted diagnostic or re-training measures. The empirical results on multiple real data sets demonstrate the proposed method's superior performance to popular explanation techniques such as Influence Function or TracIn on semantic evaluation.