Miracle-dz/UniFAR-FTUs huggingface dataset:
A Sequence Learning Model for Decision Making in the Brain

Abstract Decision making is often modelled as a competition between options. Currently, a great number of popular models to explain the accuracy and speed in decision making are based on variations of drift diffusion models (DDM), in which the options compete by accumulating evidence toward decision bounds. Attractor-based recurrent neural networks have been proposed to explain the underlying neural mechanism. Yet, it is questionable that either the DDM or attractor network is the brain’s general solution for decision making. Here, we propose an alternative recurrent neural network modeling approach based on gated recurrent units and sequence learning. Our network model is trained to learn the statistical structure of temporal sequences of sensory events, action events, and reward events. We demonstrate its learning with a reaction-time version of the weather prediction task previously studied in monkey experiments, in which both the animals’ behavior and the neuronal responses were consistent with the DDM. The network model’s performance is able to reflect the accuracy and reaction time pattern of the animals’ choice behavior. The analyses of the unit responses in the network reveal that they match important experimental findings. Notably, we find units encoding the accumulated evidence and the urgency signal. We further identify two groups of units based on their connection weights to the choice output units. Simulated lesions of each group of units produce doubly-dissociable effects on the network’s choice and reaction time behavior. Graph analyses reveal that these two groups of units belong to one highly interconnected sub-network. Finally, we show that the network is capable of making predictions consistent with the predictive coding and Bayesian inference framework. Our work offers experimentally testable predictions of how decision making is achieved in the brain. It provides an approach that may piece together experimental findings of decision making, reinforcement learning, and predictive coding. In particular, it suggests that the DDM may be a manifestation of a more general computational mechanism in the brain.
How Does Shareholder Governance Affect the Cost of Borrowing?

This paper examines the effect of internal corporate governance on creditors by implementing a regression discontinuity design. Using voting results on close-call shareholder-sponsored governance proposals, we compare the change in debt contract terms of firms with proposals passed to those with proposals failed. We find that banks demand higher loan spreads and more general covenants after the passage of proposals. The effects on loan terms are more pronounced for ex ante risky firms. Moreover, firms with proposals passed by a small margin become more volatile after the vote, indicating an increase in risk-shifting incentives of the firms. Collectively, our findings show that an increase in shareholder rights can exacerbate shareholder-debtholder conflicts and raise the cost of bank loans for firms.
Effect of Solar Concentration on the Thermodynamic Power Conversion Efficiency of Quantum-Dot Solar Cells Exhibiting Multiple Exciton Generation

Thermodynamic calculations show that all solar cells can convert solar photons into electricity or fuel with higher theoretical power conversion efficiencies under concentrated sunlight. For conventional (viz, present day) single-junction solar cells that produce at most one electron–hole pair per absorbed photon, the theoretical increase in efficiency is relatively small (absolute values of 38% at 500× vs 33% at 1×). However, when solar concentration is combined with multiple exciton generation (MEG) in semiconductor quantum dots, the increase in theoretical power conversion efficiency is greatly enhanced. For the ideal MEG case, where the threshold for exciton multiplication is twice the bandgap, Eg, the maximum thermodynamic efficiency increases to 75% at 500×, but the optimum Eg shifts to smaller values. If Eg is fixed at the 1-sun optimal level, then the maximum theoretical efficiency still increases markedly, becoming 62% at 500× for the staircase MEG characteristic (defined as producing N electron–.
Carbon surface : Area, porosity, chemistry, and energy

Abstract Data from the adsorption of nitrogen at 77 K are interpreted in terms of surface area and porosity, for a range of carbon blacks and two porous carbons, using the BET, αs, and Dubinin–Radushkevich methods. Heat of immersion (ΔHi) data, for nonpolar (n-heptane) and polar (water) liquids are correlated with surface structure and chemistry, respectively. An average value of ΔHi≈ 107 mJ m−2is obtained for the dispersion interaction of nonporous carbon blacks by immersion inton-heptane. Where nitrogen adsorption data indicate the presence of microporosity, higher values of ΔHiare observed which are thought to contain a contribution due to enhanced adsorption forces within pores. Data from samples of one specific black, which had been oxidized to varying levels using an oxygen/ozone fluidized bed apparatus, are presented showing a linear relationship between the surface oxygen levels and their polar interaction with water.
Distributed Subnetworks of Depression Defined by Direct Intracranial Neurophysiology

Quantitative biological substrates of depression remain elusive. We carried out this study to determine whether application of a novel computational approach to high spatiotemporal resolution direct neural recordings may unlock the functional organization and coordinated activity patterns of depression networks. We identified two subnetworks conserved across the majority of individuals studied. The first was characterized by left temporal lobe hypoconnectivity and pathological beta activity. The second was characterized by a hypoactive, but hyperconnected left frontal cortex. These findings identify distributed circuit activity associated with depression, link neural activity with functional connectivity profiles, and inform strategies for personalized targeted intervention.
Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection

Recent works within machine learning have been tackling inputs of ever increasing size, with cyber security presenting sequence classification problems of particularly extreme lengths. In the case of Windows executable malware detection, an input executable could be >=100 MB, which would translate to a time series with T=100,000,000 steps. To date, the closest approach to handling such task is MalConv --- a convolutional neural network capable of processing T=2,000,000 steps. Because the memory used by CNNs is O(T), this has prevented many from processing all executables or further extending the MalConv approach. In this work, we develop a new approach to temporal max pooling that makes the required memory invariant to the sequence length T. This makes MalConv 116x more memory efficient, and up to 25.8x faster to train, while removing the input length restrictions to MalConv. We re-invest these gains into improving the MalConv architecture by developing a new Global Channel Gating design, giving us an attention mechanism capable of learning feature interactions across 100 million time steps in an efficient manner, a capability lacked by the original MalConv approach.
Advanced NOMA Assisted Semi-Grant-Free Transmission Schemes for Randomly Distributed Users

Non-orthogonal multiple access (NOMA) assisted semi-grant-free (SGF) transmission has recently received significant research attention due to its outstanding ability of serving grant-free (GF) users with grant-based (GB) users’ spectrum, which greatly improves the spectrum efficiency and effectively relieves the massive access problem of 5G and beyond networks. In this paper, we first study the outage performance of the greedy best user scheduling SGF scheme (BU-SGF) by considering the impacts of Rayleigh fading, path loss, and random user locations. In order to tackle the admission fairness problem of the BU-SGF scheme, we propose a fair SGF scheme by applying cumulative distribution function (CDF)-based scheduling (CS-SGF), in which the GF user with the best channel relative to its own statistics will be admitted. Moreover, by employing the theories of order statistics and stochastic geometry, the outage performances of both BU-SGF and CS-SGF schemes are analyzed. Theoretical results show that both schemes can achieve full diversity orders only when the served users’ data rate is capped, which severely limits the rate performance of SGF schemes. To further address this issue, we propose a distributed power control strategy to relax such data rate constraint, and derive analytical expressions of the two schemes’ outage performances under this strategy. Finally, simulation results validate the fairness performance of the proposed CS-SGF scheme, the effectiveness of the power control strategy, and the accuracy of the theoretical analyses.
Speech Enhancement with Zero-Shot Model Selection

Recent research on speech enhancement (SE) has seen the emergence of deep-learning-based methods. It is still a challenging task to determine the effective ways to increase the generalizability of SE under diverse test conditions. In this study, we combine zero-shot learning and ensemble learning to propose a zero-shot model selection (ZMOS) approach to increase the generalization of SE performance. The proposed approach is realized in the offline and online phases. The offline phase clusters the entire set of training data into multiple subsets and trains a specialized SE model (termed component SE model) with each subset. The online phase selects the most suitable component SE model to perform the enhancement. Furthermore, two selection strategies were developed: selection based on the quality score (QS) and selection based on the quality embedding (QE). Both QS and QE were obtained using a Quality-Net, a non-intrusive quality assessment network. Experimental results confirmed that the proposed ZMOS approach can achieve better performance in both seen and unseen noise types compared to the baseline systems and other model selection systems, which indicates the effectiveness of the proposed approach in providing robust SE performance.
Sensitive Data Detection with High-Throughput Neural Network Models for Financial Institutions

Named Entity Recognition has been extensively investigated in many fields. However, the application of sensitive entity detection for production systems in financial institutions has not been well explored due to the lack of publicly available, labeled datasets. In this paper, we use internal and synthetic datasets to evaluate various methods of detecting NPI (Nonpublic Personally Identifiable) information commonly found within financial institutions, in both unstructured and structured data formats. Character-level neural network models including CNN, LSTM, BiLSTM-CRF, and CNN-CRF are investigated on two prediction tasks: (i) entity detection on multiple data formats, and (ii) column-wise entity prediction on tabular datasets. We compare these models with other standard approaches on both real and synthetic data, with respect to F1-score, precision, recall, and throughput. The real datasets include internal structured data and public email data with manually tagged labels. Our experimental results show that the CNN model is simple yet effective with respect to accuracy and throughput and thus, is the most suitable candidate model to be deployed in the production environment(s). Finally, we provide several lessons learned on data limitations, data labelling and the intrinsic overlap of data entities.
Structure formation limits on axion-like dark matter

We derive structure formation limits on dark matter (DM) composed of keV-scale axion-like particles (ALPs), produced via freeze-in through the interactions with photons and Standard Model (SM) fermions. We employ Lyman-alpha (Ly-α) forest data sets as well as the observed number of Milky Way (MW) subhalos. We compare results obtained using Maxwell-Boltzmann and quantum statistics for describing the SM bath. It should be emphasized that the presence of logarithmic divergences complicates the calculation of the production rate, which can not be parameterized with a simple power law behaviour. The obtained results, in combination with X-ray bounds, exclude the possibility for a photophilic “frozen-in” ALP DM with mass below ∼ 19. For the photophobic ALP scenario, in which DM couples primarily to SM fermions, the ALP DM distribution function is peaked at somewhat lower momentum and hence for such realization we find weaker limits on DM mass. Future facilities, such as the upcoming Vera C. Rubin observatory, will provide measurements with which the current bounds can be significantly improved to ∼ 80.
InSRL: A Multi-view Learning Framework Fusing Multiple Information Sources for Distantly-supervised Relation Extraction

Distant supervision makes it possible to automatically label bags of sentences for relation extraction by leveraging knowledge bases, but suffers from the sparse and noisy bag issues. Additional information sources are urgently needed to supplement the training data and overcome these issues. In this paper, we introduce two widely-existing sources in knowledge bases, namely entity descriptions, and multi-grained entity types to enrich the distantly supervised data. We see information sources as multiple views and fusing them to construct an intact space with sufficient information. An end-to-end multi-view learning framework is proposed for relation extraction via Intact Space Representation Learning (InSRL), and the representations of single views are jointly learned simultaneously. Moreover, inner-view and cross-view attention mechanisms are used to highlight important information on different levels on an entity-pair basis. The experimental results on a popular benchmark dataset demonstrate the necessity of additional information sources and the effectiveness of our framework. We will release the implementation of our model and dataset with multiple information sources after the anonymized review phase.
Quantum statistics effects and fluctuations of particle numbers near the critical point of nuclear matter

Equation of state with quantum statistics corrections is derived for a multi-component gas of particles interacting through the repulsive and attractive van der Waals (vdW) forces up to first few orders over a small parameter $\delta \approx \hbar^3 n(mT)^{-3/2}[g(1- bn)]^{-1}$, where $n$ and $T$ are the particle number density and temperature, $m$ and $g$ the particle mass and degeneracy factor. The parameter $b$ corresponds to the vdW excluded volume. For interacting system of Fermi nucleon and Bose $\alpha$ particles, a small impurity of $\alpha$ particles to the nucleon system at leading first order in both $\alpha $ particle and nucleon small parameters $\delta $ does not change much the basic results for the symmetric nuclear matter. Quantum statistics corrections lead to a dramatic behavior of fluctuations of particle number $\omega$ near the critical temperature and particle density in the case of symmetric nuclear matter. The isothermal in-compressibility $\mathcal{K}(n,T)$ which determines these fluctuations $\omega$ can be well approximated by the main first non-zero terms in its expansion as function of the both $n$ and $T$ variables in a small vicinity of the critical point. Our approximate analytical results appear to be in good agreement with the accurate numerical calculations.
MIX : a Multi-task Learning Approach to Solve Open-Domain Question Answering

In this paper, we introduce MIX : a multi-task deep learning approach to solve Open-Domain Question Answering. First, we design our system as a multi-stage pipeline made of 3 building blocks : a BM25-based Retriever, to reduce the search space; RoBERTa based Scorer and Extractor, to rank retrieved documents and extract relevant spans of text respectively. Eventually, we further improve computational efficiency of our system to deal with the scalability challenge : thanks to multi-task learning, we parallelize the close tasks solved by the Scorer and the Extractor. Our system outperforms previous state-of-the-art by 12 points in both f1-score and exact-match on the squad-open benchmark.
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning

Matrix factorization is a simple and natural test-bed to investigate the implicit regularization of gradient descent. Gunasekar et al. (2018) conjectured that Gradient Flow with infinitesimal initialization converges to the solution that minimizes the nuclear norm, but a series of recent papers argued that the language of norm minimization is not sufficient to give a full characterization for the implicit regularization. In this work, we provide theoretical and empirical evidence that for depth-2 matrix factorization, gradient flow with infinitesimal initialization is mathematically equivalent to a simple heuristic rank minimization algorithm, Greedy Low-Rank Learning, under some reasonable assumptions. This generalizes the rank minimization view from previous works to a much broader setting and enables us to construct counter-examples to refute the conjecture from Gunasekar et al. (2018). We also extend the results to the case where depth $\ge 3$, and we show that the benefit of being deeper is that the above convergence has a much weaker dependence over initialization magnitude so that this rank minimization is more likely to take effect for initialization with practical scale.
Detection of data drift and outliers affecting machine learning model performance over time

A trained ML model is deployed on another `test' dataset where target feature values (labels) are unknown. Drift is distribution change between the training and deployment data, which is concerning if model performance changes. For a cat/dog image classifier, for instance, drift during deployment could be rabbit images (new class) or cat/dog images with changed characteristics (change in distribution). We wish to detect these changes but can't measure accuracy without deployment data labels. We instead detect drift indirectly by nonparametrically testing the distribution of model prediction confidence for changes. This generalizes our method and sidesteps domain-specific feature representation. We address important statistical issues, particularly Type-1 error control in sequential testing, using Change Point Models (CPMs; see Adams and Ross 2012). We also use nonparametric outlier methods to show the user suspicious observations for model diagnosis, since the before/after change confidence distributions overlap significantly. In experiments to demonstrate robustness, we train on a subset of MNIST digit classes, then insert drift (e.g., unseen digit class) in deployment data in various settings (gradual/sudden changes in the drift proportion). A novel loss function is introduced to compare the performance (detection delay, Type-1 and 2 errors) of a drift detector under different levels of drift class contamination.
The Log Product Formula in Quantum K-theory

Abstract We prove a formula expressing the K-theoretic log Gromov-Witten invariants of a product of log smooth varieties $V \times W$ in terms of the invariants of V and W. The proof requires introducing log virtual fundamental classes in K-theory and verifying their various functorial properties. We introduce a log version of K-theory and prove the formula there as well.
Clustering with Iterated Linear Optimization

We introduce a novel method for clustering using a semidefinite programming (SDP) relaxation of the Max k-Cut problem. The approach is based on a new methodology for rounding the solution of an SDP using iterated linear optimization. We show the vertices of the Max k-Cut SDP relaxation correspond to partitions of the data into at most k sets. We also show the vertices are attractive fixed points of iterated linear optimization. We interpret the process of fixed point iteration with linear optimization as repeated relaxations of the closest vertex problem. Our experiments show that using fixed point iteration for rounding the Max k-Cut SDP relaxation leads to significantly better results when compared to randomized rounding.
On the Structure of Graphite Fluoride

The structure of graphite fluoride, (C2F)n has been investigated by X-ray analyses, solid state 19F-n.m. r. , and electron microscopy for well characterized and crystallized samples obtained from natural graphite or HOPG (highly oriented pyrolytic graphite). On the basis of the present results and structural properties derived from previous works, (C2F)n has a layered structure of stage-2 which belongs hexagonal to the system with C3h symmetry. Detailed discussions on the symmetry both for (CF)n and (C2F)n have led to possible stacking sequences each unit cell of graphite fluoride should require. The ideal structure of (C2F)n is a hexagonal crystal lattice with a = b = 2.5 A; c = 16.2 A, and a plausible stacking sequence of AB/B′A′/ with Ic (identity period) = 8.09 A. The layered structure of (CF)n is of stage-1 with A/A′/ stacking sequence. Zur Struktur von Graphitfluorid Die Struktur von Graphitfluorid, (C2F)n, wurde mittels Rontgenanalyse, 19F-Festkorper-NMR und Elektronenmikroskopie an gut charakterisierten und kristallisierten Proben von naturlichem Graphit und HOPG (highly oriented pyrolytic graphite) untersucht. Aufgrund dieser Ergebnisse und den in vorhergehenden Arbeiten erhaltenen strukturellen Eigenschaften hat (C2F)n eine Schichtstruktur der 2. Stufe, die zum hexagonalen System mit der Symmetrie C3h gehort. Detallierte Diskussionen zur Symmetrie sowohl fur (CF)n wie fur (C2F)n haben zur moglichen Stapelsequenz gefuhrt, die jede Einheitszelle des Graphitfluorids erfordern sollte. Die Idealstruktur ist ein hexagonales Kristallgitter mit a = b 2,5 A; c = 16,2 A und einer plausiblen Stapelsequenz von AB/B′A′/ mit der Identitatsperiode Ic = 8,09 A. Die Schichtstruktur von (CF)n ist von 1. Stufe mit der Stapelsequenz A/A′/.
On entropies of block-gluing subshifts

A subshift $X$ is called $c$-block gluing if for any integer $n\geq c$ and any two blocks $u$ and $v$ from the language of $X$ there exists an element of $X$ which has occurrences of $u$ and $v$ at distance $n$. In this note we study the topological entropies of $c$-block gluing binary one-dimensional subshifts. We define the set $R_c$ to be the set of entropies of all $c$-block-gluing subshifts, and $R=\cup_{c\in \mathbb{N}} R_c$. We show that the set $R$ is dense, while $R_1$ and $R_2$ are not; in particular, they have isolated points. We conjecture that the same holds for any $c$.
On $\ell_p$-Gaussian-Grothendieck problem

For $p\geq 1$ and $(g_{ij})_{1\leq i,j\leq n}$ being a matrix of i.i. d. standard Gaussian entries, we study the $n$-limit of the $\ell_p$-Gaussian-Grothendieck problem defined as \begin{align*}\max\Bigl\{\sum_{i,j=1}^n g_{ij}x_ix_j: x\in \mathbb{R}^n,\sum_{i=1}^n |x_i|^p=1\Bigr\}. \end{align*} The case $p=2$ corresponds to the top eigenvalue of the Gaussian Orthogonal Ensemble; when $p=\infty$, the maximum value is essentially the ground state energy of the Sherrington-Kirkpatrick mean-field spin glass model and its limit can be expressed by the famous Parisi formula. In the present work, we focus on the cases $1\leq p<2$ and $2<p<\infty. $ For the former, we compute the limit of the $\ell_p$-Gaussian-Grothendieck problem and investigate the structure of the set of all near optimizers along with stability estimates. In the latter case, we show that this problem admits a Parisi-type variational representation and the corresponding optimizer is weakly delocalized in the sense that its entries vanish uniformly in a polynomial order.