Miracle-dz/UniFAR-FTUs huggingface dataset:
Visual Context-aware Convolution Filters for Transformation-invariant Neural Network

We propose a novel visual context-aware filter generation module which incorporates contextual information present in images into Convolutional Neural Networks (CNNs). In contrast to traditional CNNs, we do not employ the same set of learned convolution filters for all input image instances. Our proposed input-conditioned convolution filters when combined with techniques inspired by Multi-instance learning and max-pooling, results in a transformation-invariant neural network. We investigated the performance of our proposed framework on three MNIST variations, which covers both rotation and scaling variance, and achieved 1.13% error on MNIST-rot-12k, 1.12% error on Half-rotated MNIST and 0.68% error on Scaling MNIST, which is significantly better than the state-of-the-art results. We make use of visualization to further prove the effectiveness of our visual context-aware convolution filters. Our proposed visual context-aware convolution filter generation framework can also serve as a plugin for any CNN based architecture and enhance its modeling capacity.
Generalized Pattern Avoidance via Hypergraph Containers.

We consider the problem of bounding the number of permutations $\sigma\in S_n$ that avoid a fixed permutation $\pi\in S_k$ in specific indices given by a $k$-uniform hypergraph $\Lambda$. We obtain relatively sharp bounds in the case where $\Lambda$ is a random hypergraph, and find bounds in the case where $\Lambda$ contains many large cliques.
Dynamic Network Embeddings for Network Evolution Analysis

Network embeddings learn to represent nodes as low-dimensional vectors to preserve the proximity between nodes and communities of the network for network analysis. The temporal edges (e.g., relationships, contacts, and emails) in dynamic networks are important for network evolution analysis, but few existing methods in network embeddings can capture the dynamic information from temporal edges. In this paper, we propose a novel dynamic network embedding method to analyze evolution patterns of dynamic networks effectively. Our method uses random walk to keep the proximity between nodes and applies dynamic Bernoulli embeddings to train discrete-time network embeddings in the same vector space without alignments to preserve the temporal continuity of stable nodes. We compare our method with several state-of-the-art methods by link prediction and evolving node detection, and the experiments demonstrate that our method generally has better performance in these tasks. Our method is further verified by two real-world dynamic networks via detecting evolving nodes and visualizing their temporal trajectories in the embedded space.
Machine-learning Regression of Stellar Effective Temperatures in the Second Gaia Data Release

This paper reports on the application of the supervised machine-learning algorithm to the stellar effective temperature regression for the second Gaia data release, based on the combination of the stars in four spectroscopic surveys: the Large Sky Area Multi-Object Fiber Spectroscopic Telescope, Sloan Extension for Galactic Understanding and Exploration, the Apache Point Observatory Galactic Evolution Experiment, and the Radial Velocity Extension. This combination, of about four million stars, enables us to construct one of the largest training samples for the regression and further predict reliable stellar temperatures with a rms error of 191 K. This result is more precise than that given by the Gaia second data release that is based on about sixty thousands stars. After a series of data cleaning processes, the input features that feed the regressor are carefully selected from the Gaia parameters, including the colors, the 3D position, and the proper motion. These Gaia parameters are used to predict effective temperatures for 132,739,323 valid stars in the second Gaia data release. We also present a new method for blind tests and a test for external regression without additional data. The machine-learning algorithm fed with the parameters only in one catalog provides us with an effective approach to maximize the sample size for prediction, and this methodology has a wide application prospect in future studies of astrophysics.
Algebraic Statistics in Practice: Applications to Networks

Algebraic statistics uses tools from algebra (especially from multilinear algebra, commutative algebra, and computational algebra), geometry, and combinatorics to provide insight into knotty problems in mathematical statistics. In this review, we illustrate this on three problems related to networks: network models for relational data, causal structure discovery, and phylogenetics. For each problem, we give an overview of recent results in algebraic statistics, with emphasis on the statistical achievements made possible by these tools and their practical relevance for applications to other scientific disciplines.
Coherent modulation of the electron temperature and electron–phonon couplings in a 2D material

Significance The interactions between electrons and lattice vibrations called phonons not only determine the properties of materials, but also give rise to remarkable phenomena such as superconductivity or spatial ordering of the charges and lattice. Here, we use a femtosecond laser to excite and spatially relocate electrons in TaSe2. In all observations to date, these excited electrons cool monotonically by scattering with phonons. Surprisingly, we observe an oscillation of the electron temperature that is coupled with the lattice breathing mode, and a switching in the dominant electron–phonon interactions by varying the laser excitation. Our results demonstrate a promising way to manipulate electron–phonon interactions, opening up routes to steer a quantum material toward a desired state using light. Ultrashort light pulses can selectively excite charges, spins, and phonons in materials, providing a powerful approach for manipulating their properties. Here we use femtosecond laser pulses to coherently manipulate the electron and phonon distributions, and their couplings, in the charge-density wave (CDW) material 1T-TaSe2. After exciting the material with a femtosecond pulse, fast spatial smearing of the laser-excited electrons launches a coherent lattice breathing mode, which in turn modulates the electron temperature. This finding is in contrast to all previous observations in multiple materials to date, where the electron temperature decreases monotonically via electron–phonon scattering. By tuning the laser fluence, the magnitude of the electron temperature modulation changes from ∼200 K in the case of weak excitation, to ∼1,000 K for strong laser excitation. We also observe a phase change of π in the electron temperature modulation at a critical fluence of 0.7 mJ/cm2, which suggests a switching of the dominant coupling mechanism between the coherent phonon and electrons. Our approach opens up routes for coherently manipulating the interactions and properties of two-dimensional and other quantum materials using light.
Densely Connected Search Space for More Flexible Neural Architecture Search

Neural architecture search (NAS) has dramatically advanced the development of neural network design. We revisit the search space design in most previous NAS methods and find the number and widths of blocks are set manually. However, block counts and block widths determine the network scale (depth and width) and make a great influence on both the accuracy and the model cost (FLOPs/latency). In this paper, we propose to search block counts and block widths by designing a densely connected search space, i.e., DenseNAS. The new search space is represented as a dense super network, which is built upon our designed routing blocks. In the super network, routing blocks are densely connected and we search for the best path between them to derive the final architecture. We further propose a chained cost estimation algorithm to approximate the model cost during the search. Both the accuracy and model cost are optimized in DenseNAS. For experiments on the MobileNetV2-based search space, DenseNAS achieves 75.3% top-1 accuracy on ImageNet with only 361MB FLOPs and 17.9ms latency on a single TITAN-XP. The larger model searched by DenseNAS achieves 76.1% accuracy with only 479M FLOPs. DenseNAS further promotes the ImageNet classification accuracies of ResNet-18, -34 and -50-B by 1.5%, 0.5% and 0.3% with 200M, 600M and 680M FLOPs reduction respectively. The related code is available at https://github.com/JaminFong/DenseNAS.
A Novel Deep Transfer Learning Method for Detection of Myocardial Infarction

Myocardial infarction (MI), also known as a cardiac attack, is one of the common cardiac disorders occurs when one or more coronary arteries are blocked. Hence, early detection of MI is critical for the reduction of the rising of the death rate. The cardiologists use the electrocardiogram (ECG) as a diagnostic tool to monitor and reveal the MI signals. However, all the MI signals are not constant and noisy, so it is tough to detect or observe these signals manually. Several computer-aided diagnosis systems (CADs) have been suggested to solve these difficulties. In this paper, we have proposed an effective CAD system to detect MI signals using the two-dimensional convolution neural network (CNN). In this study, we have employed two ways of the transfer learning technique to retrain the pre-trained VGG-Net and obtained two new networks VGG-MI1 and VGG-MI2. Moreover, the heartbeat data augmentation techniques are employed to increase the classification performance. We have utilized two-second ECG signals from the PTB database, which has been widely employed in MI detection studies. In case of using VGG-MI1, we achieved an accuracy, sensitivity, and specificity of 99.02%, 98.76%, and 99.17% respectively and we achieved an accuracy of 99.22%, a sensitivity of 99.15%, and a specificity of 99.49% when using VGG-MI2. Results showed that the proposed algorithm is more efficient than the state-of-the-art methods in terms of accuracy sensitivity, and specificity. Finally, the proposed algorithm can assist the specialists to detect the MI signals more precisely.
Certainty Equivalence is Efficient for Linear Quadratic Control

We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system and the cost incurred by using the optimal LQ controller enjoys a fast statistical rate, scaling as the square of the parameter error. To the best of our knowledge, our result is the first sub-optimality guarantee in the partially observed Linear Quadratic Gaussian (LQG) setting. Furthermore, in the fully observed Linear Quadratic Regulator (LQR), our result improves upon recent work by Dean et al. (2017), who present an algorithm achieving a sub-optimality gap linear in the parameter error. A key part of our analysis relies on perturbation bounds for discrete Riccati equations. We provide two new perturbation bounds, one that expands on an existing result from Konstantinov et al. (1993), and another based on a new elementary proof strategy.
Gradient based sample selection for online continual learning

A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal. Previous works often depend on task boundary and i.i. d. assumptions to properly select samples for the replay buffer. In this work, we formulate sample selection as a constraint reduction problem based on the constrained optimization view of continual learning. The goal is to select a fixed subset of constraints that best approximate the feasible region defined by the original constraints. We show that it is equivalent to maximizing the diversity of samples in the replay buffer with parameters gradient as the feature. We further develop a greedy alternative that is cheap and efficient. The advantage of the proposed method is demonstrated by comparing to other alternatives under the continual learning setting. Further comparisons are made against state of the art methods that rely on task boundaries which show comparable or even better results for our method.
Scalable Differentially Private Generative Student Model via PATE

Recent rapid development of machine learning is largely due to algorithmic breakthroughs, computation resource development, and especially the access to a large amount of training data. However, though data sharing has the great potential of improving machine learning models and enabling new applications, there have been increasing concerns about the privacy implications of data collection. In this work, we present a novel approach for training differentially private data generator G-PATE. The generator can be used to produce synthetic datasets with strong privacy guarantee while preserving high data utility. Our approach leverages generative adversarial nets (GAN) to generate data and protect data privacy based on the Private Aggregation of Teacher Ensembles (PATE) framework. Our approach improves the use of privacy budget by only ensuring differential privacy for the generator, which is the part of the model that actually needs to be published for private data generation. To achieve this, we connect a student generator with an ensemble of teacher discriminators. We also propose a private gradient aggregation mechanism to ensure differential privacy on all the information that flows from the teacher discriminators to the student generator. We empirically show that the G-PATE significantly outperforms prior work on both image and non-image datasets.
How is density compensation created in the ocean mixed layer

This study examines the impact of turbulent mixing on horizontal density compensation in the upper ocean. A series of simulations model the role of mixing in scenarios initialized with geostrophically-adjusted compensated and uncompensated thermohaline gradients. Numerical experiments isolate the influence of mixing on these gradients using idealized conditions with zero surface heat and momentum flux. Prompted by theoretical considerations and observed consequences of mixing, a new non-linear horizontal diffusion scheme is introduced as an alternative to the standard Laplacian diffusion. Results suggest that when horizontal mixing is parameterized using constant diffusivities, horizontal density compensation is substantially unchanged as the gradients erode. Simulations using the new scheme, which parameterizes mixing with horizontal diffusivities scaled by squared buoyancy gradient, suggest that horizontal mixing can produce compensated gradients during this decay, but only at scales of 10 km and less. Reducing vertical mixing to small background values has a similar effect, increasing the degree of compensation at submesoscales. Reproducing observed compensated thermohaline variability in the mixed layer at scales greater than 10 km requires external forcing. These results show important influences of mixing on density compensation within the ageostrophic submesoscale regime. In the transition from a horizontally compensated mixed layer to a partially compensated thermocline, advection must play an important role; mixing alone is insufficient.
Reserve Pricing in Repeated Second-Price Auctions with Strategic Bidders

We study revenue optimization learning algorithms for repeated second-price auctions with reserve where a seller interacts with multiple strategic bidders each of which holds a fixed private valuation for a good and seeks to maximize his expected future cumulative discounted surplus. We propose a novel algorithm that has strategic regret upper bound of $O(\log\log T)$ for worst-case valuations. This pricing is based on our novel transformation that upgrades an algorithm designed for the setup with a single buyer to the multi-buyer case. We provide theoretical guarantees on the ability of a transformed algorithm to learn the valuation of a strategic buyer, which has uncertainty about the future due to the presence of rivals.
Neutrino spin-flavor oscillations in solar environment

We study the phenomenon of neutrino spin-flavor oscillations due to solar magnetic fields. This allows us to examine how significantly the electron neutrinos produced in the solar interior undergo a resonant spin-flavor conversion. We construct analytical models for the solar magnetic field in all the three regions of the Sun. Neutrino spin-flavor oscillations in this magnetic field are examined by studying the level crossing phenomenon and comparing the two cases of zero and non-zero vacuum mixing respectively. Results from the Borexino experiment are used to place an upper limit on the magnetic field in the solar core. Related phenomena such as effects of matter on neutrino spin transitions and differences between Dirac and Majorana transitions in the solar magnetic fields are also discussed.
Identify and understand pay-it-forward reciprocity using millions of online red packets

Pay-it-forward reciprocity encourages the spread of prosocial behavior. However, existing empirical evidence of pay-it-forward behavior has been largely based on laboratory experiments, which are limited in sample size and external validity. Extending this research, our study uses a natural experiment to examine pay-it-forward reciprocity in a real-life context with a large-scale dataset of 3.4 million users of an online platform. Our natural experiment is enabled by the randomness in the mechanism that WeChat, a Chinese online social networking platform, uses to split an online monetary gift (also known as a "red packet") to its recipients. Our results show that recipients on average pay forward 10.34% of the amount they receive. We further find that "Luckiest draw" recipients, or those who obtain the largest shares of their corresponding red packets, are 1.5 times more likely to pay it forward than other recipients. Our analyses indicate that in a multiple recipient setting, users' pay-it-forward behavior is enforced by a group norm that luckiest draw recipients should send the first subsequent gift and promoted by their distributional social preferences of the random amounts split by the platform. Finally, our study shows that those recipients without any in-group friends do pay it forward, even though their pay-it-forward behavior is less likely to be driven by their reputational concerns among acquaintances. Overall, our work provides insights into mechanisms and conditions that encourage pay-it-forward reciprocity, which have implications for fostering prosocial behavior.
Sulfur-bearing Species Tracing the Disk/Envelope System in the Class I Protostellar Source Elias 29

We observed the Class I protostellar source Elias 29 with the Atacama Large Millimeter/submillimeter Array. We detected CS, SO, 34SO, SO2, and SiO line emissions in a compact component concentrated near the protostar, and a ridge component separated from the protostar by 4″ (∼500 au). The former component is abundant in SO and SO2, but deficient in CS. The abundance ratio SO/CS is as high as at the protostar, which is even higher than that in the outflow-shocked region of L1157 B1. However, organic molecules (HCOOCH3, CH3OCH3, CCH, and c-C3H2) are deficient in Elias 29. We attribute this deficiency in organic molecules and richness in SO and SO2 to the evolved nature of the source or the relatively high dust temperature (≳20 K) in the parent cloud of Elias 29. The SO and SO2 emissions trace rotation around the protostar. Assuming a highly inclined configuration (i ≥ 65°; 0° for a face-on configuration) and Keplerian motion for simplicity, the protostellar mass is estimated to be (0.8–1.0) . The 34SO and SO2 emissions are asymmetric in their spectra; the blueshifted components are weaker than the redshifted ones. Although this may be attributed to the asymmetric molecular distribution, other possibilities are also discussed.
Integrating Neural Networks Into the Blind Deblurring Framework to Compete With the End-to-End Learning-Based Methods

Recently, the end-to-end learning-based methods have been proven effective for the blind image deblurring. Without human-made assumptions or numerical algorithms, they are able to restore images with fewer artifacts and better perceptual quality. However, in practice, these methods suffer from limited performance under complex motion scenario and produces unnatural results sometimes. In this paper, in order to overcome their limitations, we propose to integrate deep convolution neural networks into a conventional deblurring framework. Specifically, we propose Stacked Estimation Residual Net (SEN) to estimate the motion flow map and Recurrent Prior Generative and Adversarial Net (RP-GAN) to learn the implicit image prior for the optimization. Comparing with the state-of-the-art end-to-end learning-based methods, the proposed method restores image content more naturally and shows better generalization ability.
Semi-parametric Realized Nonlinear Conditional Autoregressive Expectile and Expected Shortfall

A joint conditional autoregressive expectile and Expected Shortfall framework is proposed. The framework is extended through incorporating a measurement equation which models the contemporaneous dependence between the realized measures and the latent conditional expectile. Nonlinear threshold specification is further incorporated into the proposed framework. A Bayesian Markov Chain Monte Carlo method is adapted for estimation, whose properties are assessed and compared with maximum likelihood via a simulation study. One-day-ahead VaR and ES forecasting studies, with seven market indices, provide empirical support to the proposed models.
Sur-Real: Frechet Mean and Distance Transform for Complex-Valued Deep Learning

We develop a novel deep learning architecture for naturally complex valued data, which are often subject to complex scaling ambiguity. We treat each sample as a field in the space of complex numbers. With the polar form of a complex number, the general group that acts on this space is the product of planar rotation and non-zero scaling. This perspective allows us to develop not only a novel convoluation operator using weighted Fréchet mean (wFM) on a Riemannian manifold, but also to a novel fully connected layer operator using the distance to the wFM, with natural equivariant properties to non-zero scaling and planar rotations for the former and invariance properites for the latter. We demonstrate our method on widely used SAR dataset MSTAR and RadioML dataset. On MSTAR data, without any preprocessing, our network can achieve 98% classification accuracy on this highly imbalanced dataset using only ~44,000 parameters, as opposed to 94% accuracy with more than 500,000 parameters with a baseline real-valued network on the two-channel real representation of the complex valued data. On RadioML data, we got comparable classification accuracy with the baseline with only using 10% of the parameters as the baseline model.
Antisymmetric paramodular forms of weight 3

The problem of the construction of antisymmetric paramodular forms of canonical weight has been open since 1996. Any cusp form of this type determines a canonical differential form on any smooth compactification of the moduli space of Kummer surfaces associated to -polarised abelian surfaces. In this paper, we construct the first infinite family of antisymmetric paramodular forms of weight as automorphic Borcherds products whose first Fourier-Jacobi coefficient is a theta block. Bibliography: 32 titles.