Miracle-dz/UniFAR-FTUs huggingface dataset:
Common Spatial Generative Adversarial Networks based EEG Data Augmentation for Cross-Subject Brain-Computer Interface

The cross-subject application of EEG-based brain-computer interface (BCI) has always been limited by large individual difference and complex characteristics that are difficult to perceive. Therefore, it takes a long time to collect the training data of each user for calibration. Even transfer learning method pre-training with amounts of subject-independent data cannot decode different EEG signal categories without enough subject-specific data. Hence, we proposed a cross-subject EEG classification framework with a generative adversarial networks (GANs) based method named common spatial GAN (CS-GAN), which used adversarial training between a generator and a discriminator to obtain high-quality data for augmentation. A particular module in the discriminator was employed to maintain the spatial features of the EEG signals and increase the difference between different categories, with two losses for further enhancement. Through adaptive training with sufficient augmentation data, our cross-subject classification accuracy yielded a significant improvement of 15.85% than leave-one subject-out (LOO) test and 8.57% than just adapting 100 original samples on the dataset 2a of BCI competition IV. Moreover, We designed a convolutional neural networks (CNNs) based classification method as a benchmark with a similar spatial enhancement idea, which achieved remarkable results to classify motor imagery EEG data. In summary, our framework provides a promising way to deal with the cross-subject problem and promote the practical application of BCI.
Proximal Gradient Descent-Ascent: Variable Convergence under KŁ Geometry

The gradient descent-ascent (GDA) algorithm has been widely applied to solve minimax optimization problems. In order to achieve convergent policy parameters for minimax optimization, it is important that GDA generates convergent variable sequences rather than convergent sequences of function values or gradient norms. However, the variable convergence of GDA has been proved only under convexity geometries, and there lacks understanding for general nonconvex minimax optimization. This paper fills such a gap by studying the convergence of a more general proximal-GDA for regularized nonconvex-strongly-concave minimax optimization. Specifically, we show that proximal-GDA admits a novel Lyapunov function, which monotonically decreases in the minimax optimization process and drives the variable sequence to a critical point. By leveraging this Lyapunov function and the K{\L} geometry that parameterizes the local geometries of general nonconvex functions, we formally establish the variable convergence of proximal-GDA to a critical point $x^*$, i.e., $x_t\to x^*, y_t\to y^*(x^*)$. Furthermore, over the full spectrum of the K{\L}-parameterized geometry, we show that proximal-GDA achieves different types of convergence rates ranging from sublinear convergence up to finite-step convergence, depending on the geometry associated with the K{\L} parameter. This is the first theoretical result on the variable convergence for nonconvex minimax optimization.
Theoretical Constraints Imposed by Gradient Detection and Dispersal on Microbial Size in Astrobiological Environments.

The capacity to sense gradients efficiently and acquire information about the ambient environment confers many advantages such as facilitating movement toward nutrient sources or away from toxic chemicals. The amplified dispersal evinced by organisms endowed with motility is possibly beneficial in related contexts. Hence, the connections between information acquisition, motility, and microbial size are explored from an explicitly astrobiological standpoint. By using prior theoretical models, the constraints on organism size imposed by gradient detection and motility are elucidated in the form of simple heuristic scaling relations. It is argued that environments such as alkaline hydrothermal vents, which are distinguished by the presence of steep gradients, might be conducive to the existence of "small" microbes (with radii of ≳0.1 μm) in principle, when only the above two factors are considered; other biological functions (e.g., metabolism and genetic exchange) could, however, regulate the lower bound on microbial size and elevate it. The derived expressions are potentially applicable to a diverse array of settings, including those entailing solvents other than water; for example, the lakes and seas of Titan. The article concludes with a brief exposition of how this formalism may be of practical and theoretical value to astrobiology.
Max-Cut via Kuramoto-type Oscillators

We consider the Max-Cut problem. Let $G = (V,E)$ be a graph with adjacency matrix $(a_{ij})_{i,j=1}^{n}$. Burer, Monteiro&Zhang proposed to find, for $n$ angles $\left\{\theta_1, \theta_2, \dots, \theta_n\right\} \subset [0, 2\pi]$, minima of the energy $$ f(\theta_1, \dots, \theta_n) = \sum_{i,j=1}^{n} a_{ij} \cos{(\theta_i - \theta_j)}$$ because configurations achieving a global minimum leads to a partition of size 0.878 Max-Cut(G). This approach is known to be computationally viable and leads to very good results in practice. We prove that by replacing $\cos{(\theta_i - \theta_j)}$ with an explicit function $g_{\varepsilon}(\theta_i - \theta_j)$ global minima of this new functional lead to a $(1-\varepsilon)$Max-Cut(G). This suggests some interesting algorithms that perform well. It also shows that the problem of finding approximate global minima of energy functionals of this type is NP-hard in general.
Integration of ion channel proteins into a polymer matrix - investigation by the patch-clamp technique

Two types of natural ion channel proteins, gramicidin A and nicotinic acetylcholine receptor, were reconstituted into an artificial lipid/polymer matrix. The functionality of the proteins was fully maintained after the integration. Particular kinds of phospholipids and of polymer matrices had been chosen to set up the artificial mimic system, which itself constitutes a novel type biosensor. The functionality of the mimic system was verified by single channel patch-clamp measurements.
Spherical Message Passing for 3D Molecular Graphs

We consider representation learning of 3D molecular graphs in which each atom is associated with a spatial position in 3D. This is an under-explored area of research, and a principled message passing framework is currently lacking. In this work, we conduct analyses in the spherical coordinate system (SCS) for the complete identification of 3D graph structures. Based on such observations, we propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning. SMP dramatically reduces training complexity, enabling it to perform efficiently on large-scale molecules. In addition, SMP is capable of distinguishing almost all molecular structures, and the uncovered cases may not exist in practice. Based on meaningful physically-based representations of 3D information, we further propose the SphereNet for 3D molecular learning. Experimental results demonstrate that the use of meaningful 3D information in SphereNet leads to significant performance improvements in prediction tasks. Our results also demonstrate the advantages of SphereNet in terms of capability, efficiency, and scalability. Our code is publicly available as part of the DIG library (https://github.com/divelab/DIG).
Noisy Recurrent Neural Networks

We provide a general framework for studying recurrent neural networks (RNNs) trained by injecting noise into hidden states. Specifically, we consider RNNs that can be viewed as discretizations of stochastic differential equations driven by input data. This framework allows us to study the implicit regularization effect of general noise injection schemes by deriving an approximate explicit regularizer in the small noise regime. We find that, under reasonable assumptions, this implicit regularization promotes flatter minima; it biases towards models with more stable dynamics; and, in classification tasks, it favors models with larger classification margin. Sufficient conditions for global stability are obtained, highlighting the phenomenon of stochastic stabilization, where noise injection can improve stability during training. Our theory is supported by empirical results which demonstrate that the RNNs have improved robustness with respect to various input perturbations.
Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap

This paper presents a new model-free algorithm for episodic finite-horizon Markov Decision Processes (MDP), Adaptive Multi-step Bootstrap (AMB), which enjoys a stronger gap-dependent regret bound. The first innovation is to estimate the optimal $Q$-function by combining an optimistic bootstrap with an adaptive multi-step Monte Carlo rollout. The second innovation is to select the action with the largest confidence interval length among admissible actions that are not dominated by any other actions. We show when each state has a unique optimal action, AMB achieves a gap-dependent regret bound that only scales with the sum of the inverse of the sub-optimality gaps. In contrast, Simchowitz and Jamieson (2019) showed all upper-confidence-bound (UCB) algorithms suffer an additional $\Omega\left(\frac{S}{\Delta_{min}}\right)$ regret due to over-exploration where $\Delta_{min}$ is the minimum sub-optimality gap and $S$ is the number of states. We further show that for general MDPs, AMB suffers an additional $\frac{|Z_{mul}|}{\Delta_{min}}$ regret, where $Z_{mul}$ is the set of state-action pairs $(s,a)$'s satisfying $a$ is a non-unique optimal action for $s$. We complement our upper bound with a lower bound showing the dependency on $\frac{|Z_{mul}|}{\Delta_{min}}$ is unavoidable for any consistent algorithm. This lower bound also implies a separation between reinforcement learning and contextual bandits.
Output Perturbation for Differentially Private Convex Optimization with Improved Population Loss Bounds, Runtimes and Applications to Private Adversarial Training

Finding efficient, easily implementable differentially private (DP) algorithms that offer strong excess risk bounds is an important problem in modern machine learning. To date, most work has focused on private empirical risk minimization (ERM) or private population loss minimization. However, there are often other objectives--such as fairness, adversarial robustness, or sensitivity to outliers--besides average performance that are not captured in the classical ERM setup. To this end, we study a completely general family of convex, Lipschitz loss functions and establish the first known DP excess risk and runtime bounds for optimizing this broad class. We provide similar bounds under additional assumptions of smoothness and/or strong convexity. We also address private stochastic convex optimization (SCO). While $(\epsilon, \delta)$-DP ($\delta>0$) has been the focus of much recent work in private SCO, proving tight population loss bounds and runtime bounds for $(\epsilon, 0)$-DP remains a challenging open problem. We provide the tightest known $(\epsilon, 0)$-DP population loss bounds and fastest runtimes under the presence of (or lack of) smoothness and strong convexity. Our methods extend to the $\delta>0$ setting, where we offer the unique benefit of ensuring differential privacy for arbitrary $\epsilon>0$ by incorporating a new form of Gaussian noise. Finally, we apply our theory to two learning frameworks: tilted ERM and adversarial learning. In particular, our theory quantifies tradeoffs between adversarial robustness, privacy, and runtime. Our results are achieved using perhaps the simplest DP algorithm: output perturbation. Although this method is not novel conceptually, our novel implementation scheme and analysis show that the power of this method to achieve strong privacy, utility, and runtime guarantees has not been fully appreciated in prior works.
Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search

Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models. For object detection, the well-established classification and regression loss functions have been carefully designed by considering diverse learning challenges. Inspired by the recent progress in network architecture search, it is interesting to explore the possibility of discovering new loss function formulations via directly searching the primitive operation combinations. So that the learned losses not only fit for diverse object detection challenges to alleviate huge human efforts, but also have better alignment with evaluation metric and good mathematical convergence property. Beyond the previous auto-loss works on face recognition and image classification, our work makes the first attempt to discover new loss functions for the challenging object detection from primitive operation levels. We propose an effective convergence-simulation driven evolutionary search algorithm, called CSE-Autoloss, for speeding up the search progress by regularizing the mathematical rationality of loss candidates via convergence property verification and model optimization simulation. CSE-Autoloss involves the search space that cover a wide range of the possible variants of existing losses and discovers best-searched loss function combination within a short time (around 1.5 wall-clock days). We conduct extensive evaluations of loss function search on popular detectors and validate the good generalization capability of searched losses across diverse architectures and datasets. Our experiments show that the best-discovered loss function combinations outperform default combinations by 1.1% and 0.8% in terms of mAP for two-stage and one-stage detectors on COCO respectively. Our searched losses are available at https://github.com/PerdonLiu/CSE-Autoloss.
Learning State Representations from Random Deep Action-conditional Predictions

Our main contribution in this work is an empirical finding that random General Value Functions (GVFs), i.e., deep action-conditional predictions -- random both in what feature of observations they predict as well as in the sequence of actions the predictions are conditioned upon -- form good auxiliary tasks for reinforcement learning (RL) problems. In particular, we show that random deep action-conditional predictions when used as auxiliary tasks yield state representations that produce control performance competitive with state-of-the-art hand-crafted auxiliary tasks like value prediction, pixel control, and CURL in both Atari and DeepMind Lab tasks. In another set of experiments we stop the gradients from the RL part of the network to the state representation learning part of the network and show, perhaps surprisingly, that the auxiliary tasks alone are sufficient to learn state representations good enough to outperform an end-to-end trained actor-critic baseline. We opensourced our code at https://github.com/Hwhitetooth/random_gvfs.
Trends of phosphate fertilizer demand and phosphate balance in farmland soils in Japan

Abstract We analyzed the trends in phosphate (P) demand from 1966 to 1999, estimated the P flow in farmland in 1997, and evaluated the risk of environmental damage in Japan. Chemical P fertilizer demand per unit farmland area increased from 106 kg ha−1 in 1966 to 152 kg ha−1 at the end of the 1970s, then decreased 143 kg ha−1 after the mid 1980s to 117 kg ha−1 in 1999. This trend reflected recommendations on the use of large inputs to increase soil fertility until the 1970s, and the reduction of production costs since the 1990s. In 1997, the amount of P applied to farmland soils in Japan was 6.3 times higher than crop uptake. Outflow from farmland was estimated to account for only 0.6% of the applied P. Most of the applied P remained in farmland soil. P in farmland soils increased the risk of eutrophication by flooding and surface run off. Seventy six percent of applied P was in the form of chemical fertilizer. Chemical fertilizer was applied at a higher rate to fruit and nut fields, vegetable fields, and industrial crop fields than to paddy fields. The remaining P resulted in a high availability P in fruit and nut fields and horticultural fields. The amount of P in livestock excreta corresponded to half of the amount of chemical P fertilizer applied. One-third of the livestock excreta was not utilized in 1997, which was the main source of environmental load associated with agriculture. Therefore, promotion and alternative use of livestock excreta are important to reduce the environmental load, while uneven regional distribution of livestock excreta makes it more to achieve this objective difficult. Maintenance of optimal soil fertility by using chemical P fertilizer and manure is important for saving limited P resources and for securing a low environmental risk.
When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations?

We establish conditions under which gradient descent applied to fixed-width deep networks drives the logistic loss to zero, and prove bounds on the rate of convergence. Our analysis applies for smoothed approximations to the ReLU, such as Swish and the Huberized ReLU, proposed in previous applied work. We provide two sufficient conditions for convergence. The first is simply a bound on the loss at initialization. The second is a data separation condition used in prior analyses.
Near-Optimal Coding for Massive Multiple Access

We study the Gaussian multiple access channel (MAC) in the asymptotic regime where the number of users grows linearly with the codelength. We analyze coding schemes based on random linear models with approximate message passing (AMP) decoding. For fixed target error rate and number of bits per user, we obtain the exact tradeoff between energy-per-bit and the user density achievable in the large system limit. We show that a spatially coupled coding scheme with AMP decoding achieves near-optimal tradeoff for a large range of user densities. We also study the spectral efficiency versus energy-per-bit tradeoff in the regime where the number of bits per user is large.
High-energy Neutrinos from Magnetized Coronae of Active Galactic Nuclei and Prospects for Identification of Seyfert Galaxies and Quasars in Neutrino Telescopes

Particles may be accelerated in magnetized coronae via magnetic reconnections and/or plasma turbulence, leading to high-energy neutrinos and soft γ-rays. We evaluate the detectability of neutrinos from nearby bright Seyfert galaxies identified by X-ray measurements. In the disk-corona model, we find that NGC 1068 is the most promising Seyfert galaxy in the Northern sky, where IceCube is the most sensitive, and show prospects for the identification of aggregated neutrino signals from Seyfert galaxies bright in X-rays. Moreover, we demonstrate that nearby Seyfert galaxies are promising targets for the next generation of neutrino telescopes such as KM3NeT and IceCube-Gen2. For KM3NeT, Cen A can be the most promising source in the Southern sky if a significant fraction of the observed X-rays come from the corona, and it could be identified in few years of KM3NeT operation. Our results reinforce the idea that hidden cores of supermassive black holes are the dominant sources of the high-energy neutrino emission and underlines the necessity of better sensitivity to medium-energy ranges in future neutrino detectors for identifying the origin of high-energy cosmic neutrinos.
An Oversized Magnetic Sheath Wrapping around the Parsec-scale Jet in 3C 273

In recent studies, several active galactic nuclei (AGNs) have exhibited gradients of the Faraday rotation measure (RM) transverse to their parsec-scale jet direction. Faraday rotation likely occurs as a result of a magnetized sheath wrapped around the jet. In the case of 3C 273, using Very Long Baseline Array multiepoch observations at 5, 8, and 15 GHz in 2009–2010, we observe that the jet RM has changed significantly toward negative values compared with that previously observed. These changes could be explained by a swing of the parsec-scale jet direction, which causes synchrotron emission to pass through different portions of the Faraday screen. We develop a model for the jet–sheath system in 3C 273 where the sheath is wider than the single-epoch narrow relativistic jet. We present our oversized sheath model together with a derived wide-jet full intrinsic opening angle α int = 2. °1 and magnetic field strength B ∣∣ = 3 μG, and thermal particle density N e = 125 cm−3 at the wide jet–sheath boundary 230 pc downstream (deprojected) from its beginning. Most of the Faraday rotation occurs within the innermost layers of the sheath. The model brings together the jet direction swing and long-term RM evolution and may be applicable to other AGN jets that exhibit changes in their apparent jet direction.
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation

Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT-style, GPT-style, and Encoder-Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.
Target Training Does Adversarial Training Without Adversarial Samples

Neural network classifiers are vulnerable to misclassification of adversarial samples, for which the current best defense trains classifiers with adversarial samples. However, adversarial samples are not optimal for steering attack convergence, based on the minimization at the core of adversarial attacks. The minimization perturbation term can be minimized towards $0$ by replacing adversarial samples in training with duplicated original samples, labeled differently only for training. Using only original samples, Target Training eliminates the need to generate adversarial samples for training against all attacks that minimize perturbation. In low-capacity classifiers and without using adversarial samples, Target Training exceeds both default CIFAR10 accuracy ($84.3$%) and current best defense accuracy (below $25$%) with $84.8$% against CW-L$_2$($\kappa=0$) attack, and $86.6$% against DeepFool. Using adversarial samples against attacks that do not minimize perturbation, Target Training exceeds current best defense ($69.1$%) with $76.4$% against CW-L$_2$($\kappa=40$) in CIFAR10.
Determination of Trace and Minor Metals in Benign and Malign Human Thyroid Tissues

There is a need for the analysis of essential and toxic metals in cancerous and non-cancerous human thyroid tissues because the relationship between these elements and the mechanism of cancer development and inhibition were reported in the literature. Selected metal concentrations including Ni, Fe, Mg and Ca in both cancerous (malign) and non-cancerous (benign) thyroid tissues were determined by atomic absorption spectrometry. For this purpose, wet tissues were digested by using microwave energy. The calcium concentrations in the malign thyroid tissues were found to be lower significantly than in the benign thyroids. It was observed that there are slightly the reverse-proportional relationship between the changes in iron concentrations and the changes in nickel concentrations regarding cancerous and non-cancerous thyroid samples. Therefore, it is understood that the decreases in the calcium levels in the cancerous thyroid samples in compared to non-cancerous tissues and the interaction between nickel and magnesium are very important for the investigation of cancer mechanism and may be important for cancer diagnosis.