OpenRaiser/Intern-Atlas huggingface dataset:
SC-Square: Overview to 2021
Venue: arXiv cs.SC 2022
Year: 2022

This extended abstract was written to accompany an invited talk at the 2021 SC-Square Workshop, where the author was asked to give an overview of SC-Square progress to date. The author first reminds the reader of the definition of SC-Square, then briefly outlines some of the history, before picking out some (personal) scientific highlights.
SC-Square: Future Progress with Machine Learning?
Venue: arXiv cs.SC 2022
Year: 2022

The algorithms employed by our communities are often underspecified, and thus have multiple implementation choices, which do not effect the correctness of the output, but do impact the efficiency or even tractability of its production. In this extended abstract, to accompany a keynote talk at the 2021 SC-Square Workshop, we survey recent work (both the author's and from the literature) on the use of Machine Learning technology to improve algorithms of interest to SC-Square.
Private Read Update Write (PRUW) in Federated Submodel Learning (FSL): Communication Efficient Schemes With and Without Sparsification
Venue: arXiv cs.IT 2022
Year: 2022

We investigate the problem of private read update write (PRUW) in relation to private federated submodel learning (FSL), where a machine learning model is divided into multiple submodels based on the different types of data used to train the model. In PRUW, each user downloads the required submodel without revealing its index in the reading phase, and uploads the updates of the submodel without revealing the submodel index or the values of the updates in the writing phase. In this work, we first provide a basic communication efficient PRUW scheme, and study further means of reducing the communication cost via sparsification. Gradient sparsification is a widely used concept in learning applications, where only a selected set of parameters is downloaded and updated, which significantly reduces the communication cost. In this paper, we study how the concept of sparsification can be incorporated in private FSL with the goal of reducing the communication cost, while guaranteeing information theoretic privacy of the updated submodel index as well as the values of the updates. To this end, we introduce two schemes: PRUW with top $r$ sparsification and PRUW with random sparsification. The former communicates only the most significant parameters/updates among the servers and the users, while the latter communicates a randomly selected set of parameters/updates. The two proposed schemes introduce novel techniques such as parameter/update (noisy) permutations to handle the additional sources of information leakage in PRUW caused by sparsification. Both schemes result in significantly reduced communication costs compared to that of the basic (non-sparse) PRUW scheme.
FPGA Random Number Generator
Venue: arXiv cs.AR 2022
Year: 2022

Random number generation is a key technology that is useful in a variety of ways. Random numbers are often used to generate keys for data encryption. Random numbers generated at a sufficiently long length can encrypt sensitive data and make it difficult for another computer or person to decrypt the data. Other uses for random numbers include statistical sampling, search/sort algorithms, gaming, and gambling. Due to the wide array of applications for random numbers, it would be useful to create a method of generating random numbers reliably directly in hardware to generate a ready supply of a random number for whatever the end application may be. This paper offers a proof-of-concept for creating a verilog-based hardware design that utilizes random measurement and scrambling algorithms to generate 32-bit random synchronously with a single clock cycle on a field-programmable-gate-array(FPGA).
Level-set based pre-processing algorithm for particle-based methods
Venue: arXiv cs.CE 2022
Year: 2022

Obtaining high quality particle distribution representing clean geometry in pre-processing is essential for the simulation accuracy of the particle-based methods. In this paper, several level-set based techniques for cleaning up `dirty' geometries automatically and generating homogeneous particle distributions are presented. First, a non-resolved structure identifying method based on level-set field is employed to detect the tiny fragments which make the geometry `dirty' under a given resolutions. Second, a re-distance algorithm is proposed to remove the tiny fragments and reconstruct clean and smooth geometries. Third, a `static confinement' boundary condition is developed in the particle relaxation process. By complementing the kernel support for the particles near the geometric surface, the boundary condition achieves better body-fitted particle distribution on the narrow region with high curvature. Several numerical examples include a 2D airfoil 30P30N, 3D SPHinXsys symbol, a skyscraper with a flagpole and an inferior vena cava demonstrate that the present method not only cleans up the `dirty' geometries efficiently, but also provides better body-fitted homogeneous particle distribution for complex geometry.
Improving Model Training via Self-learned Label Representations
Venue: arXiv cs.LG 2022
Year: 2022

Modern neural network architectures have shown remarkable success in several large-scale classification and prediction tasks. Part of the success of these architectures is their flexibility to transform the data from the raw input representations (e.g. pixels for vision tasks, or text for natural language processing tasks) to one-hot output encoding. While much of the work has focused on studying how the input gets transformed to the one-hot encoding, very little work has examined the effectiveness of these one-hot labels. In this work, we demonstrate that more sophisticated label representations are better for classification than the usual one-hot encoding. We propose Learning with Adaptive Labels (LwAL) algorithm, which simultaneously learns the label representation while training for the classification task. These learned labels can significantly cut down on the training time (usually by more than 50%) while often achieving better test accuracies. Our algorithm introduces negligible additional parameters and has a minimal computational overhead. Along with improved training times, our learned labels are semantically meaningful and can reveal hierarchical relationships that may be present in the data.
Sparsity-guided Network Design for Frame Interpolation
Venue: arXiv cs.CV 2022
Year: 2022

DNN-based frame interpolation, which generates intermediate frames from two consecutive frames, is often dependent on model architectures with a large number of features, preventing their deployment on systems with limited resources, such as mobile devices. We present a compression-driven network design for frame interpolation that leverages model pruning through sparsity-inducing optimization to greatly reduce the model size while attaining higher performance. Concretely, we begin by compressing the recently proposed AdaCoF model and demonstrating that a 10 times compressed AdaCoF performs similarly to its original counterpart, where different strategies for using layerwise sparsity information as a guide are comprehensively investigated under a variety of hyperparameter settings. We then enhance this compressed model by introducing a multi-resolution warping module, which improves visual consistency with multi-level details. As a result, we achieve a considerable performance gain with a quarter of the size of the original AdaCoF. In addition, our model performs favorably against other state-of-the-art approaches on a wide variety of datasets. We note that the suggested compression-driven framework is generic and can be easily transferred to other DNN-based frame interpolation algorithms. The source code is available at https://github.com/tding1/CDFI.
Avoiding Pragmatic Oddity: A Bottom-up Defeasible Deontic Logic
Venue: arXiv cs.LO 2022
Year: 2022

This paper presents an extension of Defeasible Deontic Logic to deal with the Pragmatic Oddity problem. The logic applies three general principles: (1) the Pragmatic Oddity problem must be solved within a general logical treatment of CTD reasoning; (2) non-monotonic methods must be adopted to handle CTD reasoning; (3) logical models of CTD reasoning must be computationally feasible and, if possible, efficient. The proposed extension of Defeasible Deontic Logic elaborates a preliminary version of the model proposed by Governatori and Rotolo (2019). The previous solution was based on particular characteristics of the (constructive, top-down) proof theory of the logic. However, that method introduces some degree of non-determinism. To avoid the problem, we provide a bottom-up characterisation of the logic. The new characterisation offers insights for the efficient implementation of the logic and allows us to establish the computational complexity of the problem.
OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue
Venue: arXiv cs.CL 2022
Year: 2022

This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD). Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: dialogue state tracker (DST) and response generator (RG). The dialogue state consists of the domain-slot-value triples, which are regarded as the user's constraints to search the domain-related databases. The large-scale task-oriented dialogue data with the annotated structured dialogue state usually are inaccessible. It prevents the development of the pretrained language model for the task-oriented dialogue. We propose a simple yet effective pretraining method to alleviate this problem, which consists of two pretraining phases. The first phase is to pretrain on large-scale contextual text data, where the structured information of the text is extracted by the information extracting tool. To bridge the gap between the pretraining method and downstream tasks, we design two pretraining tasks: ontology-like triple recovery and next-text generation, which simulates the DST and RG, respectively. The second phase is to fine-tune the pretrained model on the TOD data. The experimental results show that our proposed method achieves an exciting boost and get competitive performance even without any TOD data on CamRest676 and MultiWOZ benchmarks.
Code Compliance Assessment as a Learning Problem
Venue: arXiv cs.SE 2022
Year: 2022

Manual code reviews and static code analyzers are the traditional mechanisms to verify if source code complies with coding policies. However, these mechanisms are hard to scale. We formulate code compliance assessment as a machine learning (ML) problem, to take as input a natural language policy and code, and generate a prediction on the code's compliance, non-compliance, or irrelevance. This can help scale compliance classification and search for policies not covered by traditional mechanisms. We explore key research questions on ML model formulation, training data, and evaluation setup. The core idea is to obtain a joint code-text embedding space which preserves compliance relationships via the vector distance of code and policy embeddings. As there is no task-specific data, we re-interpret and filter commonly available software datasets with additional pre-training and pre-finetuning tasks that reduce the semantic gap. We benchmarked our approach on two listings of coding policies (CWE and CBP). This is a zero-shot evaluation as none of the policies occur in the training set. On CWE and CBP respectively, our tool Policy2Code achieves classification accuracies of (59%, 71%) and search MRR of (0.05, 0.21) compared to CodeBERT with classification accuracies of (37%, 54%) and MRR of (0.02, 0.02). In a user study, 24% Policy2Code detections were accepted compared to 7% for CodeBERT.
Fighting Sybils in Airdrops
Venue: arXiv cs.CR 2022
Year: 2022

Airdrop is a crucial concept in tokenomics. Startups of decentralized applications (DApps) reward early supporters by airdropping newly issued tokens up to a certain amount as a free giveaway. This naturally induces greedy hackers, called Sybils, to create multiple accounts for more shares. Most airdrops have prerequisites for qualification, in which utilizing these DApps is unsurprisingly the principal. One particular characteristic of DApps is to implement users' interactions with them in the form of token transfer transactions or smart contract calling transactions on public blockchains. We argue that these individual transactions could reveal underlying signatures of their sending accounts. Specifically, accounts controlled by the same Sybil may exhibit some common behaviors. A careful analysis of Sybil's behaviors shows that accounts controlled by the same Sybil may produce similar DApp activities and regular token transfer patterns. We model the transactions as graphs by representing accounts as vertices and transactions as edges. When multiple accounts receive tokens from the same Sybil to conduct interactions with DApps, we inspect the graphs for these activities and patterns to detect suspicious accounts. We demonstrate the effectiveness of the proposed method in a recent airdrop by presenting the suspicious accounts controlled by Sybils. All the detected accounts exhibit similar interaction activities and regular transfer patterns.
LSDNet: Trainable Modification of LSD Algorithm for Real-Time Line Segment Detection
Venue: arXiv cs.CV 2022
Year: 2022

As of today, the best accuracy in line segment detection (LSD) is achieved by algorithms based on convolutional neural networks - CNNs. Unfortunately, these methods utilize deep, heavy networks and are slower than traditional model-based detectors. In this paper we build an accurate yet fast CNN- based detector, LSDNet, by incorporating a lightweight CNN into a classical LSD detector. Specifically, we replace the first step of the original LSD algorithm - construction of line segments heatmap and tangent field from raw image gradients - with a lightweight CNN, which is able to calculate more complex and rich features. The second part of the LSD algorithm is used with only minor modifications. Compared with several modern line segment detectors on standard Wireframe dataset, the proposed LSDNet provides the highest speed (among CNN-based detectors) of 214 FPS with a competitive accuracy of 78 Fh . Although the best-reported accuracy is 83 Fh at 33 FPS, we speculate that the observed accuracy gap is caused by errors in annotations and the actual gap is significantly lower. We point out systematic inconsistencies in the annotations of popular line detection benchmarks - Wireframe and York Urban, carefully reannotate a subset of images and show that (i) existing detectors have improved quality on updated annotations without retraining, suggesting that new annotations correlate better with the notion of correct line segment detection; (ii) the gap between accuracies of our detector and others diminishes to negligible 0.2 Fh , with our method being the fastest.
An EPTAS for Budgeted Matroid Independent Set
Venue: arXiv cs.DS 2022
Year: 2022

We consider the budgeted matroid independent set problem. The input is a ground set, where each element has a cost and a non-negative profit, along with a matroid over the elements and a budget. The goal is to select a subset of elements which maximizes the total profit subject to the matroid and budget constraints. Several well known special cases, where we have, e.g., a uniform matroid and a budget, or no matroid constraint (i.e., the classic knapsack problem), admit a fully polynomial-time approximation scheme (FPTAS). In contrast, already a slight generalization to the multi-budgeted matroid independent set problem has a PTAS but does not admit an efficient polynomial-time approximation scheme (EPTAS). This implies a PTAS for our problem, which is the best known result prior to this work. Our main contribution is an EPTAS for the budgeted matroid independent set problem. A key idea of the scheme is to find a representative set for the instance, whose cardinality depends solely on $1/\varepsilon$, where $\varepsilon > 0$ is the accuracy parameter of the scheme. The representative set is identified via matroid basis minimization, which can be solved by a simple greedy algorithm. Our scheme enumerates over subsets of the representative set and extends each subset using a linear program. The notion of representative sets may be useful in solving other variants of the budgeted matroid independent set problem.
Structured Q-learning For Antibody Design
Venue: arXiv cs.LG 2022
Year: 2022

Optimizing combinatorial structures is core to many real-world problems, such as those encountered in life sciences. For example, one of the crucial steps involved in antibody design is to find an arrangement of amino acids in a protein sequence that improves its binding with a pathogen. Combinatorial optimization of antibodies is difficult due to extremely large search spaces and non-linear objectives. Even for modest antibody design problems, where proteins have a sequence length of eleven, we are faced with searching over 2.05 x 10^14 structures. Applying traditional Reinforcement Learning algorithms such as Q-learning to combinatorial optimization results in poor performance. We propose Structured Q-learning (SQL), an extension of Q-learning that incorporates structural priors for combinatorial optimization. Using a molecular docking simulator, we demonstrate that SQL finds high binding energy sequences and performs favourably against baselines on eight challenging antibody design tasks, including designing antibodies for SARS-COV.
The Switch from Conventional to SDN: The Case for Transport-Agnostic Congestion Control
Venue: arXiv cs.NI 2022
Year: 2022

To meet the timing requirements of interactive applications, the no-frills congestion-agnostic transport protocols like UDP are increasingly deployed side-by-side in the same network with congestion-responsive TCP. In cloud platforms, even though the computation and storage is totally virtualized, they lack a true virtualization mechanism for the network (i.e., the underlying data centers networks). The impact of such lack of isolation services, may result into frequent outages (for some applications) when such diverse traffics contend for the small buffers in the commodity switches used in data centers. In this paper, we explore the design space of a simple, practical and transport-agnostic scheme to enable a scalable and flexible end-to-end congestion control in data centers. Then, we present the the shortcomings of coupling the monitoring and control of congestion in the conventional system and discuss how a Software-Defined Network (SDN) would provide an appealing alternative to circumvent the problems of the conventional system. The two systems implements a software-based congestion control mechanisms that perform monitoring, control decisions and traffic control enforcement functions. Both systems are designed with a major assumption that the applications (or transport protocols) are non-cooperative with the system, ultimately making it deployable in existing data centers without any service disruption or hardware upgrade. Both systems are implemented and evaluated via simulation in NS2 as well as real-life small-scale test-bed deployment and experiments.
Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysis
Venue: arXiv cs.CL 2022
Year: 2022

To what extent do pre-trained language models grasp semantic knowledge regarding the phenomenon of distributivity? In this paper, we introduce DistNLI, a new diagnostic dataset for natural language inference that targets the semantic difference arising from distributivity, and employ the causal mediation analysis framework to quantify the model behavior and explore the underlying mechanism in this semantically-related task. We find that the extent of models' understanding is associated with model size and vocabulary size. We also provide insights into how models encode such high-level semantic knowledge.
Towards Weak Source Coding
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, the authors provide a weak decoding version of the traditional source coding theorem of Claude Shannon. The central bound that is obtained is \[ χ>\log_ε(2^{-n(H(X)+ε)}) \] where \[ χ=\frac{\log(k)}{n(H(X)+ε)} \] and $k$ is the number of unsupervised learning classes formed out of the non-typical source sequences. The bound leads to the conclusion that if the number of classes is high enough, the reliability function might possibly be improved. The specific regime in which this improvement might be allowable is the one in which the atypical-sequence clusters are small in size and sparsely placed; similar regimes might also show an improvement.
SymphonyDB: A Polyglot Model for Knowledge Graph Query Processing
Venue: arXiv cs.DB 2022
Year: 2022

Unlocking the full potential of Knowledge Graphs (KGs) to enable or enhance various semantic and other applications requires Data Management Systems (DMSs) to efficiently store and process the content of KGs. However, the increases in the size and variety of KG datasets as well as the growing diversity of KG queries pose efficiency challenges for the current generation of DMSs to the extent that the performance of representative DMSs tends to vary significantly across diverse query types and no single platform dominates performance. We present our extensible prototype, SymphonyDB, as an approach to addressing this problem based on a polyglot model of query processing as part of a multi-database system supported by a unified access layer that can analyze/translate individual queries just-in-time and match each to the likely best-performing DMS among Virtuoso, Blazegraph, RDF-3X, and MongoDB as representative DMSs that are included in our prototype at this time. The results of our experiments with the prototype over well-known KG benchmark datasets and queries point to the efficiency and consistency of its performance across different query types and datasets.
Scattering Model Guided Adversarial Examples for SAR Target Recognition: Attack and Defense
Venue: arXiv cs.CV 2022
Year: 2022

Deep Neural Networks (DNNs) based Synthetic Aperture Radar (SAR) Automatic Target Recognition (ATR) systems have shown to be highly vulnerable to adversarial perturbations that are deliberately designed yet almost imperceptible but can bias DNN inference when added to targeted objects. This leads to serious safety concerns when applying DNNs to high-stake SAR ATR applications. Therefore, enhancing the adversarial robustness of DNNs is essential for implementing DNNs to modern real-world SAR ATR systems. Toward building more robust DNN-based SAR ATR models, this article explores the domain knowledge of SAR imaging process and proposes a novel Scattering Model Guided Adversarial Attack (SMGAA) algorithm which can generate adversarial perturbations in the form of electromagnetic scattering response (called adversarial scatterers). The proposed SMGAA consists of two parts: 1) a parametric scattering model and corresponding imaging method and 2) a customized gradient-based optimization algorithm. First, we introduce the effective Attributed Scattering Center Model (ASCM) and a general imaging method to describe the scattering behavior of typical geometric structures in the SAR imaging process. By further devising several strategies to take the domain knowledge of SAR target images into account and relax the greedy search procedure, the proposed method does not need to be prudentially finetuned, but can efficiently to find the effective ASCM parameters to fool the SAR classifiers and facilitate the robust model training. Comprehensive evaluations on the MSTAR dataset show that the adversarial scatterers generated by SMGAA are more robust to perturbations and transformations in the SAR processing chain than the currently studied attacks, and are effective to construct a defensive model against the malicious scatterers.