OpenRaiser/Intern-Atlas huggingface dataset:
Meta Adversarial Perturbations
Venue: arXiv cs.LG 2021
Year: 2021

A plethora of attack methods have been proposed to generate adversarial examples, among which the iterative methods have been demonstrated the ability to find a strong attack. However, the computation of an adversarial perturbation for a new data point requires solving a time-consuming optimization problem from scratch. To generate a stronger attack, it normally requires updating a data point with more iterations. In this paper, we show the existence of a meta adversarial perturbation (MAP), a better initialization that causes natural images to be misclassified with high probability after being updated through only a one-step gradient ascent update, and propose an algorithm for computing such perturbations. We conduct extensive experiments, and the empirical results demonstrate that state-of-the-art deep neural networks are vulnerable to meta perturbations. We further show that these perturbations are not only image-agnostic, but also model-agnostic, as a single perturbation generalizes well across unseen data points and different neural network architectures.
Types for Tables: A Language Design Benchmark
Venue: arXiv cs.PL 2021
Year: 2021

Context: Tables are ubiquitous formats for data. Therefore, techniques for writing correct programs over tables, and debugging incorrect ones, are vital. Our specific focus in this paper is on rich types that articulate the properties of tabular operations. We wish to study both their expressive power and _diagnostic quality_. Inquiry: There is no "standard library" of table operations. As a result, every paper (and project) is free to use its own (sub)set of operations. This makes artifacts very difficult to compare, and it can be hard to tell whether omitted operations were left out by oversight or because they cannot actually be expressed. Furthermore, virtually no papers discuss the quality of type error feedback. Approach: We combed through several existing languages and libraries to create a "standard library" of table operations. Each entry is accompanied by a detailed specification of its "type," expressed independent of (and hence not constrained by) any type language. We also studied and categorized a corpus of (student) program edits that resulted in table-related errors. We used this to generate a suite of erroneous programs. Finally, we adapted the concept of a datasheet to facilitate comparisons of different implementations. Knowledge: Our benchmark creates a common ground to frame work in this area. Language designers who claim to support typed programming over tables have a clear suite against which to demonstrate their system's expressive power. Our family of errors also gives them a chance to demonstrate the quality of feedback. Researchers who improve one aspect -- especially error reporting -- without changing the other can demonstrate their improvement, as can those who engage in trade-offs between the two. The net result should be much better science in both expressiveness and diagnostics. We also introduce a datasheet format for presenting this knowledge in a methodical way. Grounding: We have generated our benchmark from real languages, libraries, and programs, as well as personal experience conducting and teaching data science. We have drawn on experience in engineering and, more recently, in data science to generate the datasheet. Importance: Claims about type support for tabular programming are hard to evaluate. However, tabular programming is ubiquitous, and the expressive power of type systems keeps growing. Our benchmark and datasheet can help lead to more orderly science. It also benefits programmers trying to choose a language.
Spherical harmonic shape descriptors of nodal force demands for quantifying spatial truss connection complexity
Venue: arXiv cs.CE 2021
Year: 2021

The connections of a spatial truss structure play a critical role in the safe and efficient transfer of axial forces between members. For discrete connections, they can also improve construction efficiency by acting as registration devices that lock members in precise orientations. As more geometrically complex spatial trusses are enabled by computational workflows and the demand for material-efficient spanning systems, there is a need to understand the effects of global form on the demands at the connections. For large-scale structures with irregular geometry, customizing individual nodes to meet exact member orientations and force demands may be infeasible; conversely, standardizing all connections results in oversized nodes and a compromise in registration potential. We propose a method for quantifying the complexity of spatial truss designs by the variation in nodal force demands. By representing nodal forces as a geometric object, we leverage the spherical harmonic shape descriptor, developed for applications in computational geometry, to characterize each node by a rotation and translation-invariant fixed-length vector. We define a complexity score for spatial truss design by the variance in the positions of the feature vectors in higher-dimensional space, providing an additional performance metric during early stage design exploration. We then develop a pathway towards reducing complexity by clustering nodes with respect to their feature vectors to reduce the number of unique connectors for design while minimizing the effects of mass standardization.
Maximizing revenue in the presence of intermediaries
Venue: arXiv cs.GT 2021
Year: 2021

We study the mechanism design problem of selling $k$ items to unit-demand buyers with private valuations for the items. A buyer either participates directly in the auction or is represented by an intermediary, who represents a subset of buyers. Our goal is to design robust mechanisms that are independent of the demand structure (i.e. how the buyers are partitioned across intermediaries), and perform well under a wide variety of possible contracts between intermediaries and buyers. We first study the case of $k$ identical items where each buyer draws its private valuation for an item i.i.d. from a known $λ$-regular distribution. We construct a robust mechanism that, independent of the demand structure and under certain conditions on the contracts between intermediaries and buyers, obtains a constant factor of the revenue that the mechanism designer could obtain had she known the buyers' valuations. In other words, our mechanism's expected revenue achieves a constant factor of the optimal welfare, regardless of the demand structure. Our mechanism is a simple posted-price mechanism that sets a take-it-or-leave-it per-item price that depends on $k$ and the total number of buyers, but does not depend on the demand structure or the downstream contracts. Next we generalize our result to the case when the items are not identical. We assume that the item valuations are separable. For this case, we design a mechanism that obtains at least a constant fraction of the optimal welfare, by using a menu of posted prices. This mechanism is also independent of the demand structure, but makes a relatively stronger assumption on the contracts between intermediaries and buyers, namely that each intermediary prefers outcomes with a higher sum of utilities of the subset of buyers represented by it.
Data Processing Matters: SRPH-Konvergen AI's Machine Translation System for WMT'21
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we describe the submission of the joint Samsung Research Philippines-Konvergen AI team for the WMT'21 Large Scale Multilingual Translation Task - Small Track 2. We submit a standard Seq2Seq Transformer model to the shared task without any training or architecture tricks, relying mainly on the strength of our data preprocessing techniques to boost performance. Our final submission model scored 22.92 average BLEU on the FLORES-101 devtest set, and scored 22.97 average BLEU on the contest's hidden test set, ranking us sixth overall. Despite using only a standard Transformer, our model ranked first in Indonesian to Javanese, showing that data preprocessing matters equally, if not more, than cutting edge model architectures and training techniques.
Distributed CONGEST Approximation of Weighted Vertex Covers and Matchings
Venue: arXiv cs.DS 2021
Year: 2021

We provide CONGEST model algorithms for approximating minimum weighted vertex cover and the maximum weighted matching. For bipartite graphs, we show that a $(1+\varepsilon)$-approximate weighted vertex cover can be computed deterministically in polylogarithmic time. This generalizes a corresponding result for the unweighted vertex cover problem shown in [Faour, Kuhn; OPODIS '20]. Moreover, we show that in general weighted graph families that are closed under taking subgraphs and in which we can compute an independent set of weight at least a $λ$-fraction of the total weight, one can compute a $(2-2λ+\varepsilon)$-approximate weighted vertex cover in polylogarithmic time in the CONGEST model. Our result in particular implies that in graphs of arboricity $a$, one can compute a $(2-1/a+\varepsilon)$-approximate weighted vertex cover. For maximum weighted matchings, we show that a $(1-\varepsilon)$-approximate solution can be computed deterministically in polylogarithmic CONGEST rounds (for constant $\varepsilon$). We also provide a more efficient randomized algorithm. Our algorithm generalizes results of [Lotker, Patt-Shamir, Pettie; SPAA '08] and [Bar-Yehuda, Hillel, Ghaffari, Schwartzman; PODC '17] for the unweighted case. Finally, we show that even in the LOCAL model and in bipartite graphs of degree $\leq 3$, if $\varepsilon<\varepsilon_0$ for some constant $\varepsilon_0>0$, then computing a $(1+\varepsilon)$-approximation for the unweighted minimum vertex cover problem requires $Ω\big(\frac{\log n}{\varepsilon}\big)$ rounds. This generalizes aresult of [Göös, Suomela; DISC '12], who showed that computing a $(1+\varepsilon_0)$-approximation in such graphs requires $Ω(\log n)$ rounds.
Generating meta-learning tasks to evolve parametric loss for classification learning
Venue: arXiv cs.LG 2021
Year: 2021

The field of meta-learning has seen a dramatic rise in interest in recent years. In existing meta-learning approaches, learning tasks for training meta-models are usually collected from public datasets, which brings the difficulty of obtaining a sufficient number of meta-learning tasks with a large amount of training data. In this paper, we propose a meta-learning approach based on randomly generated meta-learning tasks to obtain a parametric loss for classification learning based on big data. The loss is represented by a deep neural network, called meta-loss network (MLN). To train the MLN, we construct a large number of classification learning tasks through randomly generating training data, validation data, and corresponding ground-truth linear classifier. Our approach has two advantages. First, sufficient meta-learning tasks with large number of training data can be obtained easily. Second, the ground-truth classifier is given, so that the difference between the learned classifier and the ground-truth model can be measured to reflect the performance of MLN more precisely than validation accuracy. Based on this difference, we apply the evolutionary strategy algorithm to find out the optimal MLN. The resultant MLN not only leads to satisfactory learning effects on generated linear classifier learning tasks for testing, but also behaves very well on generated nonlinear classifier learning tasks and various public classification tasks. Our MLN stably surpass cross-entropy (CE) and mean square error (MSE) in testing accuracy and generalization ability. These results illustrate the possibility of achieving satisfactory meta-learning effects using generated learning tasks.
Quality and Computation Time in Optimization Problems
Venue: arXiv cs.AI 2021
Year: 2021

Optimization problems are crucial in artificial intelligence. Optimization algorithms are generally used to adjust the performance of artificial intelligence models to minimize the error of mapping inputs to outputs. Current evaluation methods on optimization algorithms generally consider the performance in terms of quality. However, not all optimization algorithms for all test cases are evaluated equal from quality, the computation time should be also considered for optimization tasks. In this paper, we investigate the quality and computation time of optimization algorithms in optimization problems, instead of the one-for-all evaluation of quality. We select the well-known optimization algorithms (Bayesian optimization and evolutionary algorithms) and evaluate them on the benchmark test functions in terms of quality and computation time. The results show that BO is suitable to be applied in the optimization tasks that are needed to obtain desired quality in the limited function evaluations, and the EAs are suitable to search the optimal of the tasks that are allowed to find the optimal solution with enough function evaluations. This paper provides the recommendation to select suitable optimization algorithms for optimization problems with different numbers of function evaluations, which contributes to the efficiency that obtains the desired quality with less computation time for optimization problems.
A Gaussian Process-Based Ground Segmentation for Sloped Terrains
Venue: arXiv cs.RO 2021
Year: 2021

A Gaussian Process GP based ground segmentation method is proposed in this paper which is fully developed in a probabilistic framework. The proposed method tends to obtain a continuous realistic model of the ground. The LiDAR three-dimensional point cloud data is used as the sole source of the input data. The physical realities of the data are taken into account to properly classify sloped ground as well as the flat ones. Furthermore, unlike conventional ground segmentation methods, no height or distance constraints or limitations are required for the algorithm to be applied to take all the regarding physical behavior of the ground into account. Furthermore, a density-like parameter is defined to handle ground-like obstacle points in the ground candidate set. The non-stationary covariance kernel function is used for the Gaussian Process, by which Bayesian inference is applied using the maximum A Posteriori criterion. The log-marginal likelihood function is assumed to be a multi-task objective function, to represent a whole-frame unbiased view of the ground at each frame. Simulation results show the effectiveness of the proposed method even in an uneven, rough scene which outperforms similar Gaussian process-based ground segmentation methods.
Simulated LiDAR Repositioning: a novel point cloud data augmentation method
Venue: arXiv cs.CV 2021
Year: 2021

We address a data augmentation problem for LiDAR. Given a LiDAR scan of a scene from some position, how can one simulate new scans of that scene from different, secondary positions? The method defines criteria for selecting valid secondary positions, and then estimates which points from the original point cloud would be acquired by a scanner from these positions. We validate the method using synthetic scenes, and examine how the similarity of generated point clouds depends on scanner distance, occlusion, and angular resolution. We show that the method is more accurate at short distances, and that having a high scanner resolution for the original point clouds has a strong impact on the similarity of generated point clouds. We also demonstrate how the method can be applied to natural scene statistics: in particular, we apply our method to reposition the scanner horizontally and vertically, separately consider points belonging to the ground and to non-ground objects, and describe the impact on the distributions of distances to these two classes of points.
Real-time Human Detection Model for Edge Devices
Venue: arXiv cs.CV 2021
Year: 2021

Building a small-sized fast surveillance system model to fit on limited resource devices is a challenging, yet an important task. Convolutional Neural Networks (CNNs) have replaced traditional feature extraction and machine learning models in detection and classification tasks. Various complex large CNN models are proposed that achieve significant improvement in the accuracy. Lightweight CNN models have been recently introduced for real-time tasks. This paper suggests a CNN-based lightweight model that can fit on a limited edge device such as Raspberry Pi. Our proposed model provides better performance time, smaller size and comparable accuracy with existing method. The model performance is evaluated on multiple benchmark datasets. It is also compared with existing models in terms of size, average processing time, and F-score. Other enhancements for future research are suggested.
A Domain-Independent Holistic Approach to Deception Detection
Venue: arXiv cs.SI 2021
Year: 2021

The deception in the text can be of different forms in different domains, including fake news, rumor tweets, and spam emails. Irrespective of the domain, the main intent of the deceptive text is to deceit the reader. Although domain-specific deception detection exists, domain-independent deception detection can provide a holistic picture, which can be crucial to understand how deception occurs in the text. In this paper, we detect deception in a domain-independent setting using deep learning architectures. Our method outperforms the State-of-the-Art (SOTA) performance of most benchmark datasets with an overall accuracy of 93.42% and F1-Score of 93.22%. The domain-independent training allows us to capture subtler nuances of deceptive writing style. Furthermore, we analyze how much in-domain data may be helpful to accurately detect deception, especially for the cases where data may not be readily available to train. Our results and analysis indicate that there may be a universal pattern of deception lying in-between the text independent of the domain, which can create a novel area of research and open up new avenues in the field of deception detection.
The Case for Approximate Intermittent Computing
Venue: arXiv cs.AR 2021
Year: 2021

We present the concept of approximate intermittent computing and demonstrate its application. Intermittent computations stem from the erratic energy patterns caused by energy harvesting: computations unpredictably terminate whenever energy is insufficient. Existing solutions maintain equivalence to continuous executions by creating persistent state. The performance penalty is massive: system throughput reduces while energy consumption increases. Approximate intermittent computations trade the accuracy of the results for sparing the entire overhead to maintain equivalence to a continuous execution. We use approximation to limit the extent of stateful computations to the single power cycle, enabling the system to shift the energy budget for managing persistent state towards an immediate approximate result. First, we apply approximate intermittent computing to human activity recognition. We design an anytime variation of support vector machines able to improve the accuracy of the classification as energy is available. We build a hw/sw prototype using kinetic energy and show a 7x improvement in system throughput compared to state of the art, while retaining 83% accuracy in a setting where the best attainable accuracy is 88%. Next, we apply approximate intermittent computing in a sharply different scenario, that is, embedded image processing, using loop perforation. Using a different hw/sw prototype we build and diverse energy traces, we show a 5x improvement in system throughput compared to state of the art, while providing an equivalent output in 84% of the cases.
Local Linearity and Double Descent in Catastrophic Overfitting
Venue: arXiv cs.LG 2021
Year: 2021

Catastrophic overfitting is a phenomenon observed during Adversarial Training (AT) with the Fast Gradient Sign Method (FGSM) where the test robustness steeply declines over just one epoch in the training stage. Prior work has attributed this loss in robustness to a sharp decrease in $\textit{local linearity}$ of the neural network with respect to the input space, and has demonstrated that introducing a local linearity measure as a regularization term prevents catastrophic overfitting. Using a simple neural network architecture, we experimentally demonstrate that maintaining high local linearity might be $\textit{sufficient}$ to prevent catastrophic overfitting but is not $\textit{necessary.}$ Further, inspired by Parseval networks, we introduce a regularization term to AT with FGSM to make the weight matrices of the network orthogonal and study the connection between orthogonality of the network weights and local linearity. Lastly, we identify the $\textit{double descent}$ phenomenon during the adversarial training process.
The KICK-OFF of 6G Research Worldwide: An Overview
Venue: arXiv cs.IT 2021
Year: 2021

The fifth-generation (5G) mobile system is now being deployed across the world and the scale of 5G subscribers is growing quickly in many countries. The attention of academia and industry is increasingly shifting towards the sixth generation (6G) and many pioneering works are being kicked off, indicating an important milestone in the history of 6G. At this juncture, an overview of the current state of the art of 6G research and a vision of future communications are of great interest. This paper thus investigates up-to-date 6G research programs, ambitions, and main viewpoints of representative countries, institutions, and companies worldwide. Then, the key technologies are outlined and a vision on ``What 6G may look like?" is provided. This paper aims to serve as an enlightening guideline for interested researchers to quickly get an overview when kicking off their 6G research.
Experience-Enhanced Learning: One Size Still does not Fit All in Automatic Database
Venue: arXiv cs.DB 2021
Year: 2021

Recent years, the database committee has attempted to develop automatic database management systems. Although some researches show that the applying AI to data management is a significant and promising direction, there still exists many problems in implementing these techniques to real applications (long training time, various environments and unstable performance). In this paper, we discover that traditional rule based methods have the potential to solve the above problems. We propose three methodologies for improving learned methods, i.e. label collection for efficiently pre-training, knowledge base for model transfer and theoretical guarantee for stable performance. We implement our methodologies on two widely used learning approaches, deep learning and reinforcement learning. Firstly, the novel experience enhanced deep learning (EEDL) could achieve efficient training and stable performance. We evaluate EEDL with cardinality estimation, an essential database management. The experimental results on four real dataset [1] show that our EEDL could outperforms the general DL model [2]. Secondly, we design a novel experience-enhanced reinforcement learning (EERL), which could efficiently converge and has better performance than general RL models [3]. We test EERL with online index tuning task. The experiments on TPC-H shows that EERL could accelerate the convergence of agent and generate better solution that generalizes the reinforcement learning.