Miracle-dz/UniFAR-FTUs huggingface dataset:
Itsy Bitsy SpiderNet: Fully Connected Residual Network for Fraud Detection

With the development of high technology, the scope of fraud is increasing, resulting in annual losses of billions of dollars worldwide. The preventive protection measures become obsolete and vulnerable over time, so effective detective tools are needed. In this paper, we propose a convolutional neural network architecture SpiderNet designed to solve fraud detection problems. We noticed that the principles of pooling and convolutional layers in neural networks are very similar to the way antifraud analysts work when conducting investigations. Moreover, the skip-connections used in neural networks make the usage of features of various power in antifraud models possible. Our experiments have shown that SpiderNet provides better quality compared to Random Forest and adapted for antifraud modeling problems 1D-CNN, 1D-DenseNet, F-DenseNet neural networks. We also propose new approaches for fraud feature engineering called B-tests and W-tests, which generalize the concepts of Benford's Law for fraud anomalies detection. Our results showed that B-tests and W-tests give a significant increase to the quality of our anti-fraud models. The SpiderNet code is available at https://github.com/aasmirnova24/SpiderNet
Sample Efficient Linear Meta-Learning by Alternating Minimization

Meta-learning synthesizes and leverages the knowledge from a given set of tasks to rapidly learn new tasks using very little data. Meta-learning of linear regression tasks, where the regressors lie in a low-dimensional subspace, is an extensively-studied fundamental problem in this domain. However, existing results either guarantee highly suboptimal estimation errors, or require $\Omega(d)$ samples per task (where $d$ is the data dimensionality) thus providing little gain over separately learning each task. In this work, we study a simple alternating minimization method (MLLAM), which alternately learns the low-dimensional subspace and the regressors. We show that, for a constant subspace dimension MLLAM obtains nearly-optimal estimation error, despite requiring only $\Omega(\log d)$ samples per task. However, the number of samples required per task grows logarithmically with the number of tasks. To remedy this in the low-noise regime, we propose a novel task subset selection scheme that ensures the same strong statistical guarantee as MLLAM, even with bounded number of samples per task for arbitrarily large number of tasks.
Relative Positional Encoding for Transformers with Linear Complexity

Recent advances in Transformer models allow for unprecedented sequence lengths, due to linear space and time complexity. In the meantime, relative positional encoding (RPE) was proposed as beneficial for classical Transformers and consists in exploiting lags instead of absolute positions for inference. Still, RPE is not available for the recent linear-variants of the Transformer, because it requires the explicit computation of the attention matrix, which is precisely what is avoided by such methods. In this paper, we bridge this gap and present Stochastic Positional Encoding as a way to generate PE that can be used as a replacement to the classical additive (sinusoidal) PE and provably behaves like RPE. The main theoretical contribution is to make a connection between positional encoding and cross-covariance structures of correlated Gaussian processes. We illustrate the performance of our approach on the Long-Range Arena benchmark and on music generation.
Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement

Optimizing multiple competing black-box objectives is a challenging problem in many fields, including science, engineering, and machine learning. Multi-objective Bayesian optimization (MOBO) is a sample-efficient approach for identifying the optimal trade-offs between the objectives. However, many existing methods perform poorly when the observations are corrupted by noise. We propose a novel acquisition function, NEHVI, that overcomes this important practical limitation by applying a Bayesian treatment to the popular expected hypervolume improvement (EHVI) criterion and integrating over this uncertainty in the Pareto frontier. We argue that, even in the noiseless setting, generating multiple candidates in parallel is an incarnation of EHVI with uncertainty in the Pareto frontier and therefore can be addressed using the same underlying technique. Through this lens, we derive a natural parallel variant, $q$NEHVI, that reduces computational complexity of parallel EHVI from exponential to polynomial with respect to the batch size. $q$NEHVI is one-step Bayes-optimal for hypervolume maximization in both noisy and noiseless environments, and we show that it can be optimized effectively with gradient-based methods via sample average approximation. Empirically, we demonstrate not only that $q$NEHVI is substantially more robust to observation noise than existing MOBO approaches, but also that it achieves state-of-the-art optimization performance and competitive wall-times in large-batch environments.
Exemplar-Based Open-Set Panoptic Segmentation Network

We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task. This task requires performing panoptic segmentation for not only known classes but also unknown ones that have not been acknowledged during training. We investigate the practical challenges of the task and construct a benchmark on top of an existing dataset, COCO. In addition, we propose a novel exemplar-based open-set panoptic segmentation network (EOPSN) inspired by exemplar theory. Our approach identifies a new class based on exemplars, which are identified by clustering and employed as pseudoground-truths. The size of each class increases by mining new exemplars based on the similarities to the existing ones associated with the class. We evaluate EOPSN on the proposed benchmark and demonstrate the effectiveness of our proposals. The primary goal of our work is to draw the attention of the community to the recognition in the open- world scenarios. The implementation of our algorithm is available on the project webpage1.
Half-wormholes in SYK with one time point

In this note we study the SYK model with one time point, recently considered by Saad, Shenker, Stanford, and Yao. Working in a collective field description, they derived a remarkable identity: the square of the partition function with fixed couplings is well approximated by a ``wormhole'' saddle plus a ``pair of linked half-wormholes'' saddle. It explains factorization of decoupled systems. Here, we derive an explicit formula for the half-wormhole contribution. It is expressed through a hyperpfaffian of the tensor of SYK couplings. We then develop a perturbative expansion around the half-wormhole saddle. This expansion truncates at a finite order and gives the exact answer. The last term in the perturbative expansion turns out to coincide with the wormhole contribution. In this sense the wormhole saddle in this model does not need to be added separately, but instead can be viewed as a large fluctuation around the linked half-wormholes.
Generic Reversible Visible Watermarking via Regularized Graph Fourier Transform Coding

Reversible visible watermarking (RVW) is an active copyright protection mechanism. It not only transparently superimposes copyright patterns on specific positions of digital images or video frames to declare the copyright ownership information, but also completely erases the visible watermark image and thus enables restoring the original host image without any distortion. However, existing RVW algorithms mostly construct the reversible mapping mechanism for a specific visible watermarking scheme, which is not versatile. Hence, we propose a generic RVW framework to accommodate various visible watermarking schemes. In particular, we obtain a reconstruction data packet—the compressed difference image between the watermarked image and the original host image, which is embedded into the watermarked image via any conventional reversible data hiding method to facilitate the blind recovery of the host image. The key is to achieve compact compression of the difference image for efficient embedding of the reconstruction data packet. To this end, we propose regularized Graph Fourier Transform (GFT) coding, where the difference image is smoothed via the graph Laplacian regularizer for more efficient compression and then encoded by multi-resolution GFTs in an approximately optimal manner. Experimental results show that the proposed framework has much better versatility than state-of-the-art methods. Due to the small amount of auxiliary information to be embedded, the visual quality of the watermarked image is also higher.
Multi-Aspect Temporal Network Embedding: A Mixture of Hawkes Process View

Recent years have witnessed the tremendous research interests in network embedding. Extant works have taken the neighborhood formation as the critical information to reveal the inherent dynamics of network structures, and suggested encoding temporal edge formation sequences to capture the historical influences of neighbors. In this paper, however, we argue that the edge formation can be attributed to a variety of driving factors including the temporal influence, which is better referred to as multiple aspects. As a matter of fact, different node aspects can drive the formation of distinctive neighbors, giving birth to the multi-aspect embedding that relates to but goes beyond a temporal scope. Along this vein, we propose a Mixture of Hawkes-based Temporal Network Embeddings (MHNE) model to capture the aspect-driven neighborhood formation of networks. In MHNE, we encode the multi-aspect embeddings into the mixture of Hawkes processes to gain the advantages in modeling the excitation effects and the latent aspects. Specifically, a graph attention mechanism is used to assign different weights to account for the excitation effects of history events, while a Gumbel-Softmax is plugged in to derive the distribution over the aspects. Extensive experiments on 8 different temporal networks have demonstrated the great performance of the multi-aspect embeddings obtained by MHNE in comparison with the state-of-the-art methods.
Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift. Recent work has tied these shortcomings to beam search – the de facto standard inference algorithm in NMT – and Eikema & Aziz (2020) propose to use Minimum Bayes Risk (MBR) decoding on unbiased samples instead. In this paper, we empirically investigate the properties of MBR decoding on a number of previously reported biases and failure cases of beam search. We find that MBR still exhibits a length and token frequency bias, owing to the MT metrics used as utility functions, but that MBR also increases robustness against copy noise in the training data and domain shift.
Semiclassical description of electron dynamics in extended systems under intense laser fields

Mizuki Tani, 2, ∗ Tomohito Otobe, † Yasushi Shinohara, 3 and Kenichi L. Ishikawa 3, 4, ‡ Department of Nuclear Engineering and Management, Graduate School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan Kansai Photon Science Institute, National Institutes for Quantum and Radiological Science and Technology (QST), Kyoto 619-0215, Japan Photon Science Center, Graduate School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan Research Institute for Photon Science and Laser Technology The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan (Dated: May 19, 2021)
Demystifying autoparallels in alternative gravity

The equations of motion of test bodies in relativistic gravity are tightly linked to the conservation laws of the theory [1–3]. The explicit derivation of these equations has been intertwined with the development of approximation schemes within relativistic gravity [4–6]. As it is well known, geodesic curves arise as trajectories of structureless test bodies in Riemannian spacetimes with the metric gij as the gravitational field potential, that determines the metric-compatible Christoffel connection Γ̃ij k = 1 2g kl (∂igjl + ∂jgil − ∂lgij). In alternative gravity theories the set of the gravitational field variables is extended to (gij , Γij ) where the connection and the metric are no longer compatible, so that the torsion Tij k := Γij k −Γji k and the nonmetricity Qkij := −∇kgij = − ∂kgij + Γki glj + Γkj gil are nontrivial, in general. Autoparallel curves have been postulated on several occasions in the literature as candidates for the equations of motion of test bodies in alternative gravity theories. Such ad-hoc postulates, unsubstantiated by the conservation laws, usually lead to inconsistencies with the field equations. Consequently one should abstain from the practice of postulating equations of motion instead of deriving them [7, 8]. With this warning in mind, we here report on two special cases, in which autoparallel curves actually do emerge in theories with post-Riemannian spacetime structure. Let us consider the dynamics of massive particles under the action of the gravitational gij and a scalar φ field. We demonstrate that it is possible to recast the latter into a geometric property of the underlying spacetime, and construct an effective torsion Tij k and nonmetricity Qkij from this scalar field. As a preliminary step, we recall that the deviation of the spacetime geometry from the Riemannian [9] one is described by the distortion tensor
Fixed β-VAE Encoding for Curious Exploration in Complex 3D Environments

Curiosity is a general method for augmenting an environment reward with an intrinsic reward, which encourages exploration and is especially useful in sparse reward settings. As curiosity is calculated using next state prediction error, the type of state encoding used has a large impact on performance. Random features and inverse-dynamics features are generally preferred over VAEs based on previous results from Atari and other mostly 2D environments. However, unlike VAEs, they may not encode sufficient information for optimal behaviour, which becomes increasingly important as environments become more complex. In this paper, we use the sparse reward 3D physics environment Animal-AI, to demonstrate how a fixed $\beta$-VAE encoding can be used effectively with curiosity. We combine this with curriculum learning to solve the previously unsolved exploration intensive detour tasks while achieving 22\% gain in sample efficiency on the training curriculum against the next best encoding. We also corroborate the results on Atari Breakout, with our custom encoding outperforming random features and inverse-dynamics features.
SANM

Solving nonlinear systems is an important problem. Numerical continuation methods efficiently solve certain nonlinear systems. The Asymptotic Numerical Method (ANM) is a powerful continuation method that usually converges faster than Newtonian methods. ANM explores the landscape of the function by following a parameterized solution curve approximated with a high-order power series. Although ANM has successfully solved a few graphics and engineering problems, prior to our work, applying ANM to new problems required significant effort because the standard ANM assumes quadratic functions, while manually deriving the power series expansion for nonquadratic systems is a tedious and challenging task. This paper presents a novel solver, SANM, that applies ANM to solve symbolically represented nonlinear systems. SANM solves such systems in a fully automated manner. SANM also extends ANM to support many nonquadratic operators, including intricate ones such as singular value decomposition. Furthermore, SANM generalizes ANM to support the implicit homotopy form. Moreover, SANM achieves high computing performance via optimized system design and implementation. We deploy SANM to solve forward and inverse elastic force equilibrium problems and controlled mesh deformation problems with a few constitutive models. Our results show that SANM converges faster than Newtonian solvers, requires little programming effort for new problems, and delivers comparable or better performance than a hand-coded, specialized ANM solver. While we demonstrate on mesh deformation problems, SANM is generic and potentially applicable to many tasks.
X-ray bubbles in the circumgalactic medium of TNG50 Milky Way- and M31-like galaxies: signposts of supermassive black hole activity

The TNG50 cosmological simulation produces X-ray emitting bubbles, shells, and cavities in the circumgalactic gas above and below the stellar disks of Milky Way- and Andromeda-like galaxies with morphological features reminiscent of the eROSITA and Fermi bubbles in the Galaxy. Two-thirds of the 198 MW/M31 analogues inspected in TNG50 at z = 0 show one or more large-scale, coherent features of over-pressurized gas that impinge into the gaseous halo. Some of the galaxies include a succession of bubbles or shells of increasing size, ranging from a few to many tens of kpc. These are prominent in gas pressure, X-ray emission and gas temperature, and often exhibit sharp boundaries with typical shock Mach numbers of 2 − 4. The gas in the bubbles outflows with maximum (95th pctl) radial velocities of ∼100 − 1500 km s−1. TNG50 bubbles expand with speeds as high as 1000 − 2000 km s−1 (about 1 − 2 kpc Myr−1), but with a great diversity and with larger bubbles expanding at slower speeds. The bubble gas is at 106.4 − 7.2 K temperatures and is enriched to metallicities of $0.5-2~\rm Z_{\odot }$. In TNG50, the bubbles are a manifestation of episodic, kinetic, wind-like energy injections from the supermassive black holes at the galaxy centers that accrete at low Eddington ratios. According to TNG50, X-ray, and possibly γ-ray, bubbles similar to those observed in the Milky Way should be a frequent feature of disk-like galaxies prior to, or on the verge of, being quenched. They should be within the grasp of eROSITA in the local Universe.
Learning and Certification under Instance-targeted Poisoning

In this paper, we study PAC learnability and certification of predictions under instance-targeted poisoning attacks, where the adversary who knows the test instance may change a fraction of the training set with the goal of fooling the learner at the test instance. Our first contribution is to formalize the problem in various settings and to explicitly model subtle aspects such as the proper or improper nature of the learning, learner's randomness, and whether (or not) adversary's attack can depend on it. Our main result shows that when the budget of the adversary scales sublinearly with the sample complexity, (improper) PAC learnability and certification are achievable; in contrast, when the adversary's budget grows linearly with the sample complexity, the adversary can potentially drive up the expected 0-1 loss to one. We also study distribution-specific PAC learning in the same attack model and show that proper learning with certification is possible for learning half spaces under natural distributions. Finally, we empirically study the robustness of K nearest neighbour, logistic regression, multi-layer perceptron, and convolutional neural network on real data sets against targeted-poisoning attacks. Our experimental results show that many models, especially state-of-the-art neural networks, are indeed vulnerable to these strong attacks. Interestingly, we observe that methods with high standard accuracy might be more vulnerable to instance-targeted poisoning attacks.
Semi-Supervised Classification and Segmentation on High Resolution Aerial Images

FloodNet is a high-resolution image dataset acquired by a small UAV platform, DJI Mavic Pro quadcopters, after Hurricane Harvey. The dataset presents a unique challenge of advancing the damage assessment process for post-disaster scenarios using unlabeled and limited labeled dataset. We propose a solution to address their classification and semantic segmentation challenge. We approach this problem by generating pseudo labels for both classification and segmentation during training and slowly incrementing the amount by which the pseudo label loss affects the final loss. Using this semi-supervised method of training helped us improve our baseline supervised loss by a huge margin for classification, allowing the model to generalize and perform better on the validation and test splits of the dataset. In this paper, we compare and contrast the various methods and models for image classification and semantic segmentation on the FloodNet dataset.
Adversarial Examples Detection With Bayesian Neural Network

In this paper, we propose a new framework to detect adversarial examples motivated by the observations that random components can improve the smoothness of predictors and make it easier to simulate the output distribution of a deep neural network. With these observations, we propose a novel Bayesian adversarial example detector, short for BATer, to improve the performance of adversarial example detection. Specifically, we study the distributional difference of hidden layer output between natural and adversarial examples, and propose to use the randomness of the Bayesian neural network to simulate hidden layer output distribution and leverage the distribution dispersion to detect adversarial examples. The advantage of a Bayesian neural network is that the output is stochastic while a deep neural network without random components does not have such characteristics. Empirical results on several benchmark datasets against popular attacks show that the proposed BATer outperforms the state-of-the-art detectors in adversarial example detection.
Topological protection in a strongly nonlinear interface lattice

Mechanical topological insulators are well understood for linear and weakly nonlinear systems, however traditional analysis methods break down for strongly nonlinear systems since linear methods can not be applied in that case. We study one such system in the form of a one-dimensional mechanical analog of the Su-Schrieffer-Heeger interface model with strong nonlinearity of the cubic form. The frequency-energy dependence of the nonlinear bulk modes and topologically insulated mode is explored using Numerical continuation of the system's nonlinear normal modes (NNMs), and the linear stability of the NNMs are investigated using Floquet Multipliers (FMs) and Krein signature analysis. We find that the nonlinear topological lattice supports a family of topologically insulated NNMs that are parameterized by the total energy of the system and are stable within a range of frequencies. Next, it is shown that empirical calculations of the geometric Zak Phase can define an energy threshold to predict the excitability of the nonlinear topological mode, and that this threshold coincides with the energy that the topological NNM intersects the linear bulk-spectrum. These predictions are validated with numerical simulations of the nonlinear topological system. These results are also tested for parametric perturbations which preserve and break chirality in the system. Thus, we provide a new method for analyzing and predicting the existence of topologically insulated modes in a strongly nonlinear lattice based on the physical observable of band topology.