Miracle-dz/UniFAR-FTUs huggingface dataset:
Implicit Geometric Regularization for Learning Shapes

Representing shapes as level sets of neural networks has been recently proved to be useful for different shape analysis and reconstruction tasks. So far, such representations were computed using either: (i) pre-computed implicit shape representations; or (ii) loss functions explicitly defined over the neural level sets. In this paper we offer a new paradigm for computing high fidelity implicit neural representations directly from raw data (i.e., point clouds, with or without normal information). We observe that a rather simple loss function, encouraging the neural network to vanish on the input point cloud and to have a unit norm gradient, possesses an implicit geometric regularization property that favors smooth and natural zero level set surfaces, avoiding bad zero-loss solutions. We provide a theoretical analysis of this property for the linear case, and show that, in practice, our method leads to state of the art implicit neural representations with higher level-of-details and fidelity compared to previous methods.
Discriminative Adversarial Search for Abstractive Summarization

We introduce a novel approach for sequence decoding, Discriminative Adversarial Search (DAS), which has the desirable properties of alleviating the effects of exposure bias without requiring external metrics. Inspired by Generative Adversarial Networks (GANs), wherein a discriminator is used to improve the generator, our method differs from GANs in that the generator parameters are not updated at training time and the discriminator is only used to drive sequence generation at inference time. We investigate the effectiveness of the proposed approach on the task of Abstractive Summarization: the results obtained show that a naive application of DAS improves over the state-of-the-art methods, with further gains obtained via discriminator retraining. Moreover, we show how DAS can be effective for cross-domain adaptation. Finally, all results reported are obtained without additional rule-based filtering strategies, commonly used by the best performing systems available: this indicates that DAS can effectively be deployed without relying on post-hoc modifications of the generated outputs.
Geometric Step Options with Jumps: Parity Relations, PIDEs, and Semi-Analytical Pricing

The present article studies geometric step options in exponential Levy markets. Our contribution is manifold and extends several aspects of the geometric step option pricing literature. First, we provide symmetry and parity relations and derive various characterizations for both European-type and American-type geometric double barrier step options. In particular, we are able to obtain a jump-diffusion disentanglement for the early exercise premium of American-type geometric double barrier step contracts and its maturity-randomized equivalent as well as to characterize the diffusion and jump contributions to these early exercise premiums separately by means of partial integro-differential equations and ordinary integro-differential equations. As an application of our characterizations, we derive semi-analytical pricing results for (regular) European-type and American-type geometric down-and-out step call options under hyper-exponential jump-diffusion models. Lastly, we use the latter results to discuss the early exercise structure of geometric step options once jumps are added and to subsequently provide an analysis of the impact of jumps on the price and hedging parameters of (European-type and American-type) geometric step contracts.
Baryon acoustic oscillations reconstruction using convolutional neural networks

We propose a new scheme to reconstruct the baryon acoustic oscillations (BAO) signal, which contains key cosmological information, based on deep convolutional neural networks (CNN). Trained with almost no fine tuning, the network can recover large-scale modes accurately in the test set: the correlation coefficient between the true and reconstructed initial conditions reaches $90{{\ \rm per\ cent}}$ at $k\le 0.2 \, h\mathrm{Mpc}^{-1}$, which can lead to significant improvements of the BAO signal-to-noise ratio down to $k\simeq 0.4\, h\mathrm{Mpc}^{-1}$. Since this new scheme is based on the configuration-space density field in sub-boxes, it is local and less affected by survey boundaries than the standard reconstruction method, as our tests confirm. We find that the network trained in one cosmology is able to reconstruct BAO peaks in the others, i.e. recovering information lost to non-linearity independent of cosmology. The accuracy of recovered BAO peak positions is far less than that caused by the difference in the cosmology models for training and testing, suggesting that different models can be distinguished efficiently in our scheme. It is very promising that our scheme provides a different new way to extract the cosmological information from the ongoing and future large galaxy surveys.
Self-Adaptive Training: beyond Empirical Risk Minimization

We propose self-adaptive training---a new training algorithm that dynamically corrects problematic training labels by model predictions without incurring extra computational cost---to improve generalization of deep learning for potentially corrupted training data. This problem is crucial towards robustly learning from data that are corrupted by, e.g., label noises and out-of-distribution samples. The standard empirical risk minimization (ERM) for such data, however, may easily overfit noises and thus suffers from sub-optimal performance. In this paper, we observe that model predictions can substantially benefit the training process: self-adaptive training significantly improves generalization over ERM under various levels of noises, and mitigates the overfitting issue in both natural and adversarial training. We evaluate the error-capacity curve of self-adaptive training: the test error is monotonously decreasing w.r. t. model capacity. This is in sharp contrast to the recently-discovered double-descent phenomenon in ERM which might be a result of overfitting of noises. Experiments on CIFAR and ImageNet datasets verify the effectiveness of our approach in two applications: classification with label noise and selective classification. We release our code at this https URL.
Parallel Clique Counting and Peeling Algorithms

Dense subgraphs capture strong communities in social networks and entities possessing strong interactions in biological networks. In particular, $k$-clique counting and listing have applications in identifying important actors in a graph. However, finding $k$-cliques is computationally expensive, and thus it is important to have fast parallel algorithms. We present a new parallel algorithm for $k$-clique counting that has polylogarithmic span and is work-efficient with respect to the well-known sequential algorithm for $k$-clique listing by Chiba and Nishizeki. Our algorithm can be extended to support listing and enumeration, and is based on computing low out-degree orientations. We present a new linear-work and polylogarithmic span algorithm for computing such orientations, and new parallel algorithms for producing unbiased estimations of clique counts. Finally, we design new parallel work-efficient algorithms for approximating the $k$-clique densest subgraph. Our first algorithm gives a $1/k$-approximation and is based on iteratively peeling vertices with the lowest clique counts; our algorithm is work-efficient, but we prove that this process is P-complete and hence does not have polylogarithmic span. Our second algorithm gives a $1/(k(1+\epsilon))$-approximation, is work-efficient, and has polylogarithmic span. In addition, we implement these algorithms and propose optimizations. On a 60-core machine, we achieve 13.23-38.99x and 1.19-13.76x self-relative parallel speedup for $k$-clique counting and $k$-clique densest subgraph, respectively. Compared to the state-of-the-art parallel $k$-clique counting algorithms, we achieve a 1.31-9.88x speedup, and compared to existing implementations of $k$-clique densest subgraph, we achieve a 1.01-11.83x speedup. We are able to compute the $4$-clique counts on the largest publicly-available graph with over two hundred billion edges.
The bursty star formation history of the Fornax dwarf spheroidal galaxy revealed with the HST

We present a new derivation of the star formation history (SFH) of the dSph galaxy Fornax in two central regions, characterized by unprecedented precision and age resolution. It reveals that star formation has proceeded in sharp bursts separated by periods of low level or quiescent activity. The SFH was derived through colour–magnitude diagram (CMD) fitting of two extremely deep Hubble Space Telescope CMDs, sampling the centre and one core radius. The attained age resolution allowed us to single out a major star formation episode at early times, a second strong burst 4.6 ± 0.4 Gyr ago and recent intermittent episodes ∼2–0.2 Gyr ago. Detailed testing with mock stellar populations was used to estimate the duration of the main bursts and study the occurrence of low-level star formation between them. The SFHs in both regions show common features, with activity at the same epochs and similar age–metallicity relationship. However, clear indications of a spatial gradient were also found, with mean age increasing with radius and star formation episodes being more prolonged in the centre. While some galaxy evolution models predict bursty SFHs in dwarf galaxies and thus a secular origin of the observed SFH cannot be excluded in Fornax, other evidence points to possible mergers or interactions as the cause of its bursty SFH. In particular, we calculated the Fornax orbit relative to the closest dwarfs and the Milky Way and observed a correspondence between the main intermediate-age and young events and peri-passages of Fornax around the Milky Way, possibly indicating tidally induced star formation.
Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers

Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical robustness. In principle, convex relaxation can provide tight bounds if the solution to the relaxed problem is feasible for the original non-convex problem. We propose two regularizers that can be used to train neural networks that yield tighter convex relaxation bounds for robustness. In all of our experiments, the proposed regularizers result in higher certified accuracy than non-regularized baselines.
When Relation Networks meet GANs: Relation GANs with Triplet Loss

Though recent research has achieved remarkable progress in generating realistic images with generative adversarial networks (GANs), the lack of training stability is still a lingering concern of most GANs, especially on high-resolution inputs and complex datasets. Since the randomly generated distribution can hardly overlap with the real distribution, training GANs often suffers from the gradient vanishing problem. A number of approaches have been proposed to address this issue by constraining the discriminator's capabilities using empirical techniques, like weight clipping, gradient penalty, spectral normalization etc. In this paper, we provide a more principled approach as an alternative solution to this issue. Instead of training the discriminator to distinguish real and fake input samples, we investigate the relationship between paired samples by training the discriminator to separate paired samples from the same distribution and those from different distributions. To this end, we explore a relation network architecture for the discriminator and design a triplet loss which performs better generalization and stability. Extensive experiments on benchmark datasets show that the proposed relation discriminator and new loss can provide significant improvement on variable vision tasks including unconditional and conditional image generation and image translation.
Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning

Code summarization generates brief natural language description given a source code snippet, while code retrieval fetches relevant source code given a natural language query. Since both tasks aim to model the association between natural language and programming language, recent studies have combined these two tasks to improve their performance. However, researchers have yet been able to effectively leverage the intrinsic connection between the two tasks as they train these tasks in a separate or pipeline manner, which means their performance can not be well balanced. In this paper, we propose a novel end-to-end model for the two tasks by introducing an additional code generation task. More specifically, we explicitly exploit the probabilistic correlation between code summarization and code generation with dual learning, and utilize the two encoders for code summarization and code generation to train the code retrieval task via multi-task learning. We have carried out extensive experiments on an existing dataset of SQL and Python, and results show that our model can significantly improve the results of the code retrieval task over the-state-of-art models, as well as achieve competitive performance in terms of BLEU score for the code summarization task.
A Novel Decomposition Solution Approach for the Restoration Problem in Distribution Networks

The distribution network restoration problem is by nature a mixed integer and non-linear optimization problem due to the switching decisions and Optimal Power Flow (OPF) constraints, respectively. The link between these two parts involves logical implications modelled through big-M coefficients. The presence of these coefficients makes the relaxation of the mixed-integer problem using branch-and-bound method very poor in terms of computation burden. Moreover, this link inhibits the use of classical Benders algorithm in decomposing the problem because the resulting cuts will still depend on the big-M coefficients. In this paper, a novel decomposition approach is proposed for the restoration problem named Modified Combinatorial Benders (MCB). In this regard, the reconfiguration problem and the OPF problem are decomposed into master and sub problems, which are solved through successive iterations. In the case of a large outage area, the numerical results show that the MCB provides, within a short time (after a few iterations), a restoration solution with a quality that is close to the proven optimality when it can be exhibited.
Exploring the Connection Between Binary and Spiking Neural Networks

On-chip edge intelligence has necessitated the exploration of algorithmic techniques to reduce the compute requirements of current machine learning frameworks. This work aims to bridge the recent algorithmic progress in training Binary Neural Networks and Spiking Neural Networks—both of which are driven by the same motivation and yet synergies between the two have not been fully explored. We show that training Spiking Neural Networks in the extreme quantization regime results in near full precision accuracies on large-scale datasets like CIFAR-100 and ImageNet. An important implication of this work is that Binary Spiking Neural Networks can be enabled by “In-Memory” hardware accelerators catered for Binary Neural Networks without suffering any accuracy degradation due to binarization. We utilize standard training techniques for non-spiking networks to generate our spiking networks by conversion process and also perform an extensive empirical analysis and explore simple design-time and run-time optimization techniques for reducing inference latency of spiking networks (both for binary and full-precision models) by an order of magnitude over prior work. Our implementation source code and trained models are available at https://github.com/NeuroCompLab-psu/SNN-Conversion.
Geometric Algebra Power Theory (GAPoT): Revisiting Apparent Power under Non-Sinusoidal Conditions

Traditional power theories and one of their most important concepts --apparent power-- are still a source of debate and, as shown in the literature, they present several flaws that misinterpret the power-transfer phenomena under distorted grid conditions. In recent years, advanced mathematical tools such as geometric algebra (GA) have been applied to address these issues. However, the application of GA to electrical circuits requires more consensus, improvements and refinement. In this paper, power theories based on GA are revisited. Several drawbacks and inconsistencies of previous works are identified and modifications to the so-called geometric algebra power theory (GAPoT) are presented. This theory takes into account power components generated by cross-products between current and voltage harmonics in the frequency domain. Compared to other theories based on GA, it is compatible with the traditional definition of apparent power calculated as the product of RMS voltage and current. Also, mathematical developments are done in a multi-dimensional Euclidean space where the energy conservation principle is satisfied. The paper includes a basic example and experimental results in which measurements from a utility supply are analysed. Finally, suggestions for the extension to three-phase systems are drawn.
Multiple chemodynamic stellar populations of the Ursa Minor dwarf spheroidal galaxy

We present a Bayesian method to identify multiple (chemodynamic) stellar populations in dwarf spheroidal galaxies (dSphs) using velocity, metallicity, and positional stellar data without the assumption of spherical symmetry. We apply this method to a new Keck/Deep Imaging Multi-Object Spectrograph (DEIMOS) spectroscopic survey of the Ursa Minor (UMi) dSph. We identify 892 likely members, making this the largest UMi sample with line-of-sight velocity and metallicity measurements. Our Bayesian method detects two distinct chemodynamic populations with high significance (in logarithmic Bayes factor, ln B ∼ 33). The metal-rich ([Fe/H] = −2.05 ± 0.03) population is kinematically colder (radial velocity dispersion of $\sigma _v=4.9_{-1.0}^{+0.8} \, \mathrm{km} \, \mathrm{s}^{-1}$) and more centrally concentrated than the metal-poor ($[{\rm Fe/H}]=-2.29_{-0.06}^{+0.05}$) and kinematically hotter population ($\sigma _v =11.5_{-0.8}^{+0.9}\, \mathrm{km} \, \mathrm{s}^{-1}$). Furthermore, we apply the same analysis to an independent Multiple Mirror Telescope (MMT)/Hectochelle data set and confirm the existence of two chemodynamic populations in UMi. In both data sets, the metal-rich population is significantly flattened (ϵ = 0.75 ± 0.03) and the metal-poor population is closer to spherical ($\epsilon =0.33_{-0.09}^{+0.12}$). Despite the presence of two populations, we are able to robustly estimate the slope of the dynamical mass profile. We found hints for prolate rotation of order ${\sim}2 \, \mathrm{km} \, \mathrm{s}^{-1}$ in the MMT data set, but further observations are required to verify this. The flattened metal-rich population invalidates assumptions built into simple dynamical mass estimators, so we computed new astrophysical dark matter annihilation (J) and decay profiles based on the rounder, hotter metal-poor population and inferred $\log _{10}{(J(0{^{\circ}_{.}}5)/{\rm GeV^{2} \, cm^{-5}})}\approx 19.1$ for the Keck data set. Our results paint a more complex picture of the evolution of UMi than previously discussed.
Performance Analysis of Intelligent Reflecting Surface Assisted NOMA Networks

Intelligent reflecting surface (IRS) is a promising technology to enhance the coverage and performance of wireless networks. We consider the application of IRS to non-orthogonal multiple access (NOMA), where a base station transmits superposed signals to multiple users by the virtue of an IRS. The performance of an IRS-assisted NOMA networks with imperfect successive interference cancellation (ipSIC) and perfect successive interference cancellation (pSIC) is investigated by invoking 1-bit coding scheme. In particular, we derive new exact and asymptotic expressions for both outage probability and ergodic rate of the m-th user with ipSIC/pSIC. Based on analytical results, the diversity order of the m-th user with pSIC is in connection with the number of reflecting elements and channel ordering. The high signal-to-noise radio (SNR) slope of ergodic rate for the m-th user is obtained. The throughput and energy efficiency of IRS-NOMA networks are discussed both in delay-limited and delay-tolerant transmission modes. Additionally, we derive new exact expressions of outage probability and ergodic rate for IRS-assisted orthogonal multiple access (IRS-OMA). Numerical results are presented to substantiate our analyses and demonstrate that: i) The outage behaviors of IRS-NOMA are superior to that of IRS-OMA and relaying schemes; ii) The M-th user has a larger ergodic rate than IRS-OMA and benchmarks. However, the ergodic performance of the m-th user exceeds relaying schemes in the low SNR regime; and iii) The IRS-assisted NOMA networks have ability to achieve the enhanced energy efficiency compared to conventional cooperative communications.
Fast and Stable Adversarial Training through Noise Injection

Adversarial training is the most successful empirical method, to increase the robustness of neural networks against adversarial attacks yet. Unfortunately, this higher robustness is accompanied by considerably higher computational complexity. To date, only adversarial training with expensive multi-step adversarial attacks like Projected Gradient Descent (PGD) proved effective against equally strong attacks. In this paper, we present two ideas that combined enable adversarial training with the computationally less expensive Fast Gradient Sign Method (FGSM). First, we add uniform noise to the initial data point of the FGSM attack, which creates a wider variety of stronger adversaries. Further, we add a learnable regularization step prior to the neural network called Stochastic Augmentation Layer (SAL). Inputs propagated trough the SAL are resampled from a Gaussian distribution. The randomness of the resampling at inference time makes it more complicated for the attacker to construct an adversarial example since the outcome of the model is not known in advance. We show that noise injection in conjunction with FGSM adversarial training achieves comparable results to adversarial training with PGD while being orders of magnitude faster. Moreover, we show superior results in comparison to PGD-based training when combining noise injection and SAL.