OpenRaiser/Intern-Atlas huggingface dataset:
Experimental determination of Lamb wave dispersion diagrams over large frequency ranges in fiber metal laminates
Venue: arXiv cs.CE 2022
Year: 2022

Fiber metal laminates (FML) are of high interest for lightweight structures as they combine the advantageous material properties of metals and fiber-reinforced polymers. However, low-velocity impacts can lead to complex internal damage. Therefore, structural health monitoring with guided ultrasonic waves (GUW) is a methodology to identify such damage. Numerical simulations form the basis for corresponding investigations, but experimental validation of dispersion diagrams over a wide frequency range is hardly found in the literature. In this work the dispersive relation of GUWs is experimentally determined for an FML made of carbon fiber-reinforced polymer and steel. For this purpose, multi-frequency excitation signals are used to generate GUWs and the resulting wave field is measured via laser scanning vibrometry. The data are processed by means of a non-uniform discrete 2d Fourier transform and analyzed in the frequency-wavenumber domain. The experimental data are in excellent agreement with data from a numerical solution of the analytical framework. In conclusion, this work presents a highly automatable method to experimentally determine dispersion diagrams of GUWs in FML over large frequency ranges with high accuracy.
Is my transaction done yet? An empirical study of transaction processing times in the Ethereum Blockchain Platform
Venue: arXiv cs.SE 2022
Year: 2022

Ethereum is one of the most popular platforms for the development of blockchain-powered applications. These applications are known as Dapps. When engineering Dapps, developers need to translate requests captured in the front-end of their application into one or more smart contract transactions. Developers need to pay for these transactions and, the more they pay (i.e., the higher the gas price), the faster the transaction is likely to be processed. Therefore developers need to optimize the balance between cost (transaction fees) and user experience (transaction processing times). Online services have been developed to provide transaction issuers (e.g., Dapp developers) with an estimate of how long transactions will take to be processed given a certain gas price. These estimation services are crucial in the Ethereum domain and several popular wallets such as Metamask rely on them. However, their accuracy has not been empirically investigated so far. In this paper, we quantify the transaction processing times in Ethereum, investigate the relationship between processing times and gas prices, and determine the accuracy of state-of-the-practice estimation services. We find that transactions are processed in a median of 57s and that 90% of the transactions are processed within 8m. The higher gas prices result in faster transaction processing times with diminishing returns. In particular, we observe no practical difference in processing time between expensive and very expensive transactions. In terms of accuracy of processing time estimation services, we note that they are equivalent. However, when stratifying transactions by gas prices, Etherscan's Gas Tracker is the most accurate estimation service for very cheap and cheap transaction. EthGasStation's Gas Price API, in turn, is the most accurate estimation service for regular, expensive, and very expensive transactions.
Towards a Deep Multi-layered Dialectal Language Analysis: A Case Study of African-American English
Venue: arXiv cs.CL 2022
Year: 2022

Currently, natural language processing (NLP) models proliferate language discrimination leading to potentially harmful societal impacts as a result of biased outcomes. For example, part-of-speech taggers trained on Mainstream American English (MAE) produce non-interpretable results when applied to African American English (AAE) as a result of language features not seen during training. In this work, we incorporate a human-in-the-loop paradigm to gain a better understanding of AAE speakers' behavior and their language use, and highlight the need for dialectal language inclusivity so that native AAE speakers can extensively interact with NLP systems while reducing feelings of disenfranchisement.
Towards Consensus: Reducing Polarization by Perturbing Social Networks
Venue: arXiv cs.SI 2022
Year: 2022

This paper studies how a centralized planner can modify the structure of a social or information network to reduce polarization. First, polarization is found to be highly dependent on degree and structural properties of the network -- including the well-known isoperimetric number (i.e., Cheeger constant). We then formulate the planner's problem under full information, and motivate disagreement-seeking and coordinate descent heuristics. A novel setting for the planner in which the population's innate opinions are adversarially chosen is introduced, and shown to be equivalent to maximization of the Laplacian's spectral gap. We prove bounds for the effectiveness of a strategy that adds edges between vertices on opposite sides of the cut induced by the spectral gap's eigenvector. Finally, these strategies are evaluated on six real-world and synthetic networks. In several networks, we find that polarization can be significantly reduced through the addition of a small number of edges.
Towards Efficient Active Learning of PDFA
Venue: arXiv cs.FL 2022
Year: 2022

We propose a new active learning algorithm for PDFA based on three main aspects: a congruence over states which takes into account next-symbol probability distributions, a quantization that copes with differences in distributions, and an efficient tree-based data structure. Experiments showed significant performance gains with respect to reference implementations.
Comment on Transferability and Input Transformation with Additive Noise
Venue: arXiv cs.LG 2022
Year: 2022

Adversarial attacks have verified the existence of the vulnerability of neural networks. By adding small perturbations to a benign example, adversarial attacks successfully generate adversarial examples that lead misclassification of deep learning models. More importantly, an adversarial example generated from a specific model can also deceive other models without modification. We call this phenomenon ``transferability". Here, we analyze the relationship between transferability and input transformation with additive noise by mathematically proving that the modified optimization can produce more transferable adversarial examples.
Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problems
Venue: arXiv cs.LG 2022
Year: 2022

We prove fast mixing and characterize the stationary distribution of the Langevin Algorithm for inverting random weighted DNN generators. This result extends the work of Hand and Voroninski from efficient inversion to efficient posterior sampling. In practice, to allow for increased expressivity, we propose to do posterior sampling in the latent space of a pre-trained generative model. To achieve that, we train a score-based model in the latent space of a StyleGAN-2 and we use it to solve inverse problems. Our framework, Score-Guided Intermediate Layer Optimization (SGILO), extends prior work by replacing the sparsity regularization with a generative prior in the intermediate layer. Experimentally, we obtain significant improvements over the previous state-of-the-art, especially in the low measurement regime.
Certified Graph Unlearning
Venue: arXiv cs.LG 2022
Year: 2022

Graph-structured data is ubiquitous in practice and often processed using graph neural networks (GNNs). With the adoption of recent laws ensuring the ``right to be forgotten'', the problem of graph data removal has become of significant importance. To address the problem, we introduce the first known framework for \emph{certified graph unlearning} of GNNs. In contrast to standard machine unlearning, new analytical and heuristic unlearning challenges arise when dealing with complex graph data. First, three different types of unlearning requests need to be considered, including node feature, edge and node unlearning. Second, to establish provable performance guarantees, one needs to address challenges associated with feature mixing during propagation. The underlying analysis is illustrated on the example of simple graph convolutions (SGC) and their generalized PageRank (GPR) extensions, thereby laying the theoretical foundation for certified unlearning of GNNs. Our empirical studies on six benchmark datasets demonstrate excellent performance-complexity trade-offs when compared to complete retraining methods and approaches that do not leverage graph information. For example, when unlearning $20\%$ of the nodes on the Cora dataset, our approach suffers only a $0.1\%$ loss in test accuracy while offering a $4$-fold speed-up compared to complete retraining. Our scheme also outperforms unlearning methods that do not leverage graph information with a $12\%$ increase in test accuracy for a comparable time complexity.
PHN: Parallel heterogeneous network with soft gating for CTR prediction
Venue: arXiv cs.LG 2022
Year: 2022

The Click-though Rate (CTR) prediction task is a basic task in recommendation system. Most of the previous researches of CTR models built based on Wide \& deep structure and gradually evolved into parallel structures with different modules. However, the simple accumulation of parallel structures can lead to higher structural complexity and longer training time. Based on the Sigmoid activation function of output layer, the linear addition activation value of parallel structures in the training process is easy to make the samples fall into the weak gradient interval, resulting in the phenomenon of weak gradient, and reducing the effectiveness of training. To this end, this paper proposes a Parallel Heterogeneous Network (PHN) model, which constructs a network with parallel structure through three different interaction analysis methods, and uses Soft Selection Gating (SSG) to feature heterogeneous data with different structure. Finally, residual link with trainable parameters are used in the network to mitigate the influence of weak gradient phenomenon. Furthermore, we demonstrate the effectiveness of PHN in a large number of comparative experiments, and visualize the performance of the model in training process and structure.
New LP-based Upper Bounds in the Rate-vs.-Distance Problem for Linear Codes
Venue: arXiv cs.IT 2022
Year: 2022

We develop a new family of linear programs, that yield upper bounds on the rate of binary linear codes of a given distance. Our bounds apply {\em only to linear codes.} Delsarte's LP is the weakest member of this family and our LP yields increasingly tighter upper bounds on the rate as its control parameter increases. Numerical experiments show significant improvement compared to Delsarte. These convincing numerical results, and the large variety of tools available for asymptotic analysis, give us hope that our work will lead to new and improved asymptotic upper bounds on the possible rate of linear codes. A concurrent work by Coregliano, Jeronimo, and Jones offers a closely related family of linear programs which converges to the true bound. Here we provide a new proof of convergence for the same LPs.
Model-Agnostic Few-Shot Open-Set Recognition
Venue: arXiv cs.LG 2022
Year: 2022

We tackle the Few-Shot Open-Set Recognition (FSOSR) problem, i.e. classifying instances among a set of classes for which we only have few labeled samples, while simultaneously detecting instances that do not belong to any known class. Departing from existing literature, we focus on developing model-agnostic inference methods that can be plugged into any existing model, regardless of its architecture or its training procedure. Through evaluating the embedding's quality of a variety of models, we quantify the intrinsic difficulty of model-agnostic FSOSR. Furthermore, a fair empirical evaluation suggests that the naive combination of a kNN detector and a prototypical classifier ranks before specialized or complex methods in the inductive setting of FSOSR. These observations motivated us to resort to transduction, as a popular and practical relaxation of standard few-shot learning problems. We introduce an Open Set Transductive Information Maximization method OSTIM, which hallucinates an outlier prototype while maximizing the mutual information between extracted features and assignments. Through extensive experiments spanning 5 datasets, we show that OSTIM surpasses both inductive and existing transductive methods in detecting open-set instances while competing with the strongest transductive methods in classifying closed-set instances. We further show that OSTIM's model agnosticity allows it to successfully leverage the strong expressive abilities of the latest architectures and training strategies without any hyperparameter modification, a promising sign that architectural advances to come will continue to positively impact OSTIM's performances.
Automatic Summarization of Russian Texts: Comparison of Extractive and Abstractive Methods
Venue: arXiv cs.CL 2022
Year: 2022

The development of large and super-large language models, such as GPT-3, T5, Switch Transformer, ERNIE, etc., has significantly improved the performance of text generation. One of the important research directions in this area is the generation of texts with arguments. The solution of this problem can be used in business meetings, political debates, dialogue systems, for preparation of student essays. One of the main domains for these applications is the economic sphere. The key problem of the argument text generation for the Russian language is the lack of annotated argumentation corpora. In this paper, we use translated versions of the Argumentative Microtext, Persuasive Essays and UKP Sentential corpora to fine-tune RuBERT model. Further, this model is used to annotate the corpus of economic news by argumentation. Then the annotated corpus is employed to fine-tune the ruGPT-3 model, which generates argument texts. The results show that this approach improves the accuracy of the argument generation by more than 20 percentage points (63.2% vs. 42.5%) compared to the original ruGPT-3 model.
SAViR-T: Spatially Attentive Visual Reasoning with Transformers
Venue: arXiv cs.CV 2022
Year: 2022

We present a novel computational model, "SAViR-T", for the family of visual reasoning problems embodied in the Raven's Progressive Matrices (RPM). Our model considers explicit spatial semantics of visual elements within each image in the puzzle, encoded as spatio-visual tokens, and learns the intra-image as well as the inter-image token dependencies, highly relevant for the visual reasoning task. Token-wise relationship, modeled through a transformer-based SAViR-T architecture, extract group (row or column) driven representations by leveraging the group-rule coherence and use this as the inductive bias to extract the underlying rule representations in the top two row (or column) per token in the RPM. We use this relation representations to locate the correct choice image that completes the last row or column for the RPM. Extensive experiments across both synthetic RPM benchmarks, including RAVEN, I-RAVEN, RAVEN-FAIR, and PGM, and the natural image-based "V-PROM" demonstrate that SAViR-T sets a new state-of-the-art for visual reasoning, exceeding prior models' performance by a considerable margin.
Neural Shape-from-Shading for Survey-Scale Self-Consistent Bathymetry from Sidescan
Venue: arXiv cs.RO 2022
Year: 2022

Sidescan sonar is a small and cost-effective sensing solution that can be easily mounted on most vessels. Historically, it has been used to produce high-definition images that experts may use to identify targets on the seafloor or in the water column. While solutions have been proposed to produce bathymetry solely from sidescan, or in conjunction with multibeam, they have had limited impact. This is partly a result of mostly being limited to single sidescan lines. In this paper, we propose a modern, salable solution to create high quality survey-scale bathymetry from many sidescan lines. By incorporating multiple observations of the same place, results can be improved as the estimates reinforce each other. Our method is based on sinusoidal representation networks, a recent advance in neural representation learning. We demonstrate the scalability of the approach by producing bathymetry from a large sidescan survey. The resulting quality is demonstrated by comparing to data collected with a high-precision multibeam sensor.
From Universal Humanoid Control to Automatic Physically Valid Character Creation
Venue: arXiv cs.GR 2022
Year: 2022

Automatically designing virtual humans and humanoids holds great potential in aiding the character creation process in games, movies, and robots. In some cases, a character creator may wish to design a humanoid body customized for certain motions such as karate kicks and parkour jumps. In this work, we propose a humanoid design framework to automatically generate physically valid humanoid bodies conditioned on sequence(s) of pre-specified human motions. First, we learn a generalized humanoid controller trained on a large-scale human motion dataset that features diverse human motion and body shapes. Second, we use a design-and-control framework to optimize a humanoid's physical attributes to find body designs that can better imitate the pre-specified human motion sequence(s). Leveraging the pre-trained humanoid controller and physics simulation as guidance, our method is able to discover new humanoid designs that are customized to perform pre-specified human motions.
Enforcing continuous symmetries in physics-informed neural network for solving forward and inverse problems of partial differential equations
Venue: arXiv cs.LG 2022
Year: 2022

As a typical application of deep learning, physics-informed neural network (PINN) {has been} successfully used to find numerical solutions of partial differential equations (PDEs), but how to improve the limited accuracy is still a great challenge for PINN. In this work, we introduce a new method, symmetry-enhanced physics informed neural network (SPINN) where the invariant surface conditions induced by the Lie symmetries or non-classical symmetries of PDEs are embedded into the loss function in PINN, to improve the accuracy of PINN for solving the forward and inverse problems of PDEs. We test the effectiveness of SPINN for the forward problem via two groups of ten independent numerical experiments using different numbers of collocation points and neurons for the heat equation, Korteweg-de Vries (KdV) equation and potential Burgers {equations} respectively, and for the inverse problem by considering different layers and neurons as well as different training points for the Burgers equation in potential form. The numerical results show that SPINN performs better than PINN with fewer training points and simpler architecture of neural network. Furthermore, we discuss the computational overhead of SPINN in terms of the relative computational cost to PINN and show that the training time of SPINN has no obvious increases, even less than PINN for certain cases.
A Survey on Model-based Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Reinforcement learning (RL) solves sequential decision-making problems via a trial-and-error process interacting with the environment. While RL achieves outstanding success in playing complex video games that allow huge trial-and-error, making errors is always undesired in the real world. To improve the sample efficiency and thus reduce the errors, model-based reinforcement learning (MBRL) is believed to be a promising direction, which builds environment models in which the trial-and-errors can take place without real costs. In this survey, we take a review of MBRL with a focus on the recent progress in deep RL. For non-tabular environments, there is always a generalization error between the learned environment model and the real environment. As such, it is of great importance to analyze the discrepancy between policy training in the environment model and that in the real environment, which in turn guides the algorithm design for better model learning, model usage, and policy training. Besides, we also discuss the recent advances of model-based techniques in other forms of RL, including offline RL, goal-conditioned RL, multi-agent RL, and meta-RL. Moreover, we discuss the applicability and advantages of MBRL in real-world tasks. Finally, we end this survey by discussing the promising prospects for the future development of MBRL. We think that MBRL has great potential and advantages in real-world applications that were overlooked, and we hope this survey could attract more research on MBRL.
Automatic Map Generation for Autonomous Driving System Testing
Venue: arXiv cs.SE 2022
Year: 2022

High-definition (HD) maps are essential in testing autonomous driving systems (ADSs). HD maps essentially determine the potential diversity of the testing scenarios. However, the current HD maps suffer from two main limitations: lack of junction diversity in the publicly available HD maps and cost-consuming to build a new HD map. Hence, in this paper, we propose, FEAT2MAP, to automatically generate concise HD maps with scenario diversity guarantees. FEAT2MAP focuses on junctions as they significantly influence scenario diversity, especially in urban road networks. FEAT2MAP first defines a set of features to characterize junctions. Then, FEAT2MAP extracts and samples concrete junction features from a list of input HD maps or user-defined requirements. Each junction feature generates a junction. Finally, FEAT2MAP builds a map by connecting the junctions in a grid layout. To demonstrate the effectiveness of FEAT2MAP, we conduct experiments with the public HD maps from SVL and the open-source ADS Apollo. The results show that FEAT2MAP can (1) generate new maps of reduced size while maintaining scenario diversity in terms of the code coverage and motion states of the ADS under test, and (2) generate new maps of increased scenario diversity by merging intersection features from multiple maps or taking user inputs.
Sampling from Log-Concave Distributions over Polytopes via a Soft-Threshold Dikin Walk
Venue: arXiv cs.DS 2022
Year: 2022

Given a Lipschitz or smooth convex function $\, f:K \to \mathbb{R}$ for a bounded polytope $K \subseteq \mathbb{R}^d$ defined by $m$ inequalities, we consider the problem of sampling from the log-concave distribution $π(θ) \propto e^{-f(θ)}$ constrained to $K$. Interest in this problem derives from its applications to Bayesian inference and differentially private learning. Our main result is a generalization of the Dikin walk Markov chain to this setting that requires at most $O((md + d L^2 R^2) \times md^{ω-1}) \log(\frac{w}δ))$ arithmetic operations to sample from $π$ within error $δ>0$ in the total variation distance from a $w$-warm start. Here $L$ is the Lipschitz-constant of $f$, $K$ is contained in a ball of radius $R$ and contains a ball of smaller radius $r$, and $ω$ is the matrix-multiplication constant. Our algorithm improves on the running time of prior works for a range of parameter settings important for the aforementioned learning applications. Technically, we depart from previous Dikin walks by adding a "soft-threshold" regularizer derived from the Lipschitz or smoothness properties of $f$ to the log-barrier function for $K$ that allows our version of the Dikin walk to propose updates that have a high Metropolis acceptance ratio for $f$, while at the same time remaining inside the polytope $K$.