nils-herrmann/clean_scirepeval huggingface dataset:
Unsupervised Co-Learning on $\mathcal{G}$-Manifolds Across Irreducible Representations

We introduce a novel co-learning paradigm for manifolds naturally equipped with a group action, motivated by recent developments on learning a manifold from attached fibre bundle structures. We utilize a representation theoretic mechanism that canonically associates multiple independent vector bundles over a common base manifold, which provides multiple views for the geometry of the underlying manifold. The consistency across these fibre bundles provide a common base for performing unsupervised manifold co-learning through the redundancy created artificially across irreducible representations of the transformation group. We demonstrate the efficacy of the proposed algorithmic paradigm through drastically improved robust nearest neighbor search and community detection on rotation-invariant cryo-electron microscopy image analysis.
Flexible Cancer-Associated Chromatin Configuration (CACC) Might Be the Fundamental Reason Why Cancer Is So Difficult to Cure

We once proposed that cell-type-associated chromatin configurations determine cell types and that cancer cell type is determined by cancer-associated chromatin configuration (CACC). In this paper, we hypothesize that flexible cell-type-associated chromatin configuration is associated with cell potency and has an advantage over inflexible one in regulating genome related activities, such as DNA replication, DNA transcription, DNA repair, and DNA mutagenesis. The reason why cancer is so difficult to treat is because CACC is flexible, which enables cancer cells not only to produce various heterogeneous subclones through limited cell differentiation, but also to maximally and efficiently use genome related resources to survive environmental changes. Therefore, to beat cancer, more efforts should be made to restrict the flexibility of CACC or to change CACC so that cancer cells can be turned back to normal or become less malignant.
Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading

Although neural conversational models are effective in learning how to produce fluent responses, their primary challenge lies in knowing what to say to make the conversation contentful and non-vacuous. We present a new end-to-end approach to contentful neural conversation that jointly models response generation and on-demand machine reading. The key idea is to provide the conversation model with relevant long-form text on the fly as a source of external knowledge. The model performs QA-style reading comprehension on this text in response to each conversational turn, thereby allowing for more focused integration of external knowledge than has been possible in prior approaches. To support further research on knowledge-grounded conversation, we introduce a new large-scale conversation dataset grounded in external web pages (2.8M turns, 7.4M sentences of grounding). Both human evaluation and automated metrics show that our approach results in more contentful responses compared to a variety of previous methods, improving both the informativeness and diversity of generated output.
Mesh R-CNN

Rapid advances in 2D perception have led to systems that accurately detect objects in real-world images. However, these systems make predictions in 2D, ignoring the 3D structure of the world. Concurrently, advances in 3D shape prediction have mostly focused on synthetic benchmarks and isolated objects. We unify advances in these two areas. We propose a system that detects objects in real-world images and produces a triangle mesh giving the full 3D shape of each detected object. Our system, called Mesh R-CNN, augments Mask R-CNN with a mesh prediction branch that outputs meshes with varying topological structure by first predicting coarse voxel representations which are converted to meshes and refined with a graph convolution network operating over the mesh's vertices and edges. We validate our mesh prediction branch on ShapeNet, where we outperform prior work on single-image shape prediction. We then deploy our full Mesh R-CNN system on Pix3D, where we jointly detect objects and predict their 3D shapes.
The relationship between black hole mass and galaxy properties: examining the black hole feedback model in IllustrisTNG

Supermassive black hole feedback is thought to be responsible for the lack of star formation, or quiescence, in a significant fraction of galaxies. We explore how observable correlations between the specific star formation rate (sSFR), stellar mass (Mstar), and black hole mass (MBH) are sensitive to the physics of black hole feedback in a galaxy formation model. We use the IllustrisTNG simulation suite, specifically the TNG100 simulation and 10 model variations that alter the parameters of the black hole model. Focusing on central galaxies at z = 0 with Mstar > 1010 M⊙, we find that the sSFR of galaxies in IllustrisTNG decreases once the energy from black hole kinetic winds at low accretion rates becomes larger than the gravitational binding energy of gas within the galaxy stellar radius. This occurs at a particular MBH threshold above which galaxies are found to sharply transition from being mostly star forming to mostly quiescent. As a result of this behaviour, the fraction of quiescent galaxies as a function of Mstar is sensitive to both the normalization of the MBH–Mstar relation and the MBH threshold for quiescence in IllustrisTNG. Finally, we compare these model results to observations of 91 central galaxies with dynamical MBH measurements with the caveat that this sample is not representative of the whole galaxy population. While IllustrisTNG reproduces the observed trend that quiescent galaxies host more massive black holes, the observations exhibit a broader scatter in MBH at a given Mstar and show a smoother decline in sSFR with MBH.
Planck scale origin of nonzero 
θ13
 and super-WIMP dark matter

We study a discrete flavour symmetric scenario for neutrino mass and dark matter under the circumstances where such global discrete symmetries can be explicitly broken at Planck scale, possibly by gravitational effects. Such explicit breaking of discrete symmetries mimic as Planck suppressed operators in the model which can have non-trivial consequences for neutrino and dark matter sectors. In particular, we study a flavour symmetric model which, at renormalisable level, gives rise to tri-bimaximal type neutrino mixing with vanishing reactor mixing angle $\theta_{13}=0$, a stable inert scalar doublet behaving like a weakly interacting massive particle (WIMP) and a stable singlet inert fermion which does not interact with any other particles. The introduction of Planck suppressed operators which explicitly break the discrete symmetries, can give rise to the generation of non-zero $\theta_{13}$ in agreement with neutrino data and also open up decay channels of inert scalar doublet into singlet neutral inert fermion leading to the realisation of the super-WIMP dark matter scenario. We show that the correct neutrino phenomenology can be obtained in this model while discussing three distinct realisation of the super-WIMP dark matter scenario.
Gauge-invariant bounce from loop quantum gravity

We present a gauge-invariant treatment of singularity resolution using loop quantum gravity techniques with respect to local SU(2) transformations. Our analysis reveals many novel features of quantum geometry which were till now hidden in models based on non-gauge-invariant discretizations. Quantum geometric effects resolve the big bang singularity replacing it with a non-singular bounce when spacetime curvature reaches Planckian value. The bounce is found to be generically asymmetric in the sense that pre-bounce and post-bounce branches are not mirrored to each other and effective constants, such as Newton’s constant, are rescaled across the bounce. Furthermore, in the vicinity of the bounce, minimally coupled matter behaves as non-minimally coupled. These ramifications of quantum geometry open a rich avenue for potential phenomenological signatures.
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View

The Transformer architecture is widely used in natural language processing. Despite its success, the design principle of the Transformer remains elusive. In this paper, we provide a novel perspective towards understanding the architecture: we show that the Transformer can be mathematically interpreted as a numerical Ordinary Differential Equation (ODE) solver for a convection-diffusion equation in a multi-particle dynamic system. In particular, how words in a sentence are abstracted into contexts by passing through the layers of the Transformer can be interpreted as approximating multiple particles' movement in the space using the Lie-Trotter splitting scheme and the Euler's method. Given this ODE's perspective, the rich literature of numerical analysis can be brought to guide us in designing effective structures beyond the Transformer. As an example, we propose to replace the Lie-Trotter splitting scheme by the Strang-Marchuk splitting scheme, a scheme that is more commonly used and with much lower local truncation errors. The Strang-Marchuk splitting scheme suggests that the self-attention and position-wise feed-forward network (FFN) sub-layers should not be treated equally. Instead, in each layer, two position-wise FFN sub-layers should be used, and the self-attention sub-layer is placed in between. This leads to a brand new architecture. Such an FFN-attention-FFN layer is "Macaron-like", and thus we call the network with this new architecture the Macaron Net. Through extensive experiments, we show that the Macaron Net is superior to the Transformer on both supervised and unsupervised learning tasks. The reproducible codes and pretrained models can be found at this https URL
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP

The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization being present rather than by helping the optimization process (Frankle & Carbin, 2019). Intriguingly, this phenomenon suggests that initialization strategies for DNNs can be improved substantially, but the lottery ticket hypothesis has only previously been tested in the context of supervised learning for natural image tasks. Here, we evaluate whether "winning ticket" initializations exist in two different domains: natural language processing (NLP) and reinforcement learning (RL).For NLP, we examined both recurrent LSTM models and large-scale Transformer models (Vaswani et al., 2017). For RL, we analyzed a number of discrete-action space tasks, including both classic control and pixel control. Consistent with workin supervised image classification, we confirm that winning ticket initializations generally outperform parameter-matched random initializations, even at extreme pruning rates for both NLP and RL. Notably, we are able to find winning ticket initializations for Transformers which enable models one-third the size to achieve nearly equivalent performance. Together, these results suggest that the lottery ticket hypothesis is not restricted to supervised learning of natural images, but rather represents a broader phenomenon in DNNs.
Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policies

Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empirical gains. The choice of policy distribution, a factored Gaussian, is motivated by \cut{chosen due}its easy re-parametrization rather than its modeling power. We introduce Normalizing Flow policies within the SAC framework that learn more expressive classes of policies than simple factored Gaussians. \cut{We also present a series of stabilization tricks that enable effective training of these policies in the RL setting.}We show empirically on continuous grid world tasks that our approach increases stability and is better suited to difficult exploration in sparse reward settings.
Learning in Gated Neural Networks

Gating is a key feature in modern neural networks including LSTMs, GRUs and sparsely-gated deep neural networks. The backbone of such gated networks is a mixture-of-experts layer, where several experts make regression decisions and gating controls how to weigh the decisions in an input-dependent manner. Despite having such a prominent role in both modern and classical machine learning, very little is understood about parameter recovery of mixture-of-experts since gradient descent and EM algorithms are known to be stuck in local optima in such models. 
In this paper, we perform a careful analysis of the optimization landscape and show that with appropriately designed loss functions, gradient descent can indeed learn the parameters accurately. A key idea underpinning our results is the design of two {\em distinct} loss functions, one for recovering the expert parameters and another for recovering the gating parameters. We demonstrate the first sample complexity results for parameter recovery in this model for any algorithm and demonstrate significant performance gains over standard loss functions in numerical experiments.
Benchmarking 6D Object Pose Estimation for Robotics

Benchmarking 6D object pose estimation for robotics is not straightforward as sufficient accuracy depends on many factors, e.g., the selected gripper, dimensions, weight and material of an object, grasping point, and the robot task itself. We formulate the problem as a “successful grasp”, i.e. for a fixed set of factors affecting the task, will the given pose estimate provide sufficiently good grasp to complete the task. The successful grasp is modelled in a probabilistic framework by sampling in the pose error space and executing the task and automatically detecting success or failure. Hours of sampling and thousands of samples are used to construct a non-parametric probability of a successful grasp given the pose residual. The framework is experimentally validated with real objects and assembly tasks and comparison of several state-ofthe-art point cloud based 3D pose estimation methods.
A novel approach to model exploration for value function learning

Planning and Learning are complementary approaches. Planning relies on deliberative reasoning about the current state and sequence of future reachable states to solve the problem. Learning, on the other hand, is focused on improving system performance based on experience or available data. Learning to improve the performance of planning based on experience in similar, previously solved problems, is ongoing research. One approach is to learn Value function (cost-to-go) which can be used as heuristics for speeding up search-based planning. Existing approaches in this direction use the results of the previous search for learning the heuristics. In this work, we present a search-inspired approach of systematic model exploration for the learning of the value function which does not stop when a plan is available but rather prolongs search such that not only resulting optimal path is used but also extended region around the optimal path. This, in turn, improves both the efficiency and robustness of successive planning. Additionally, the effect of losing admissibility by using ML heuristic is managed by bounding ML with other admissible heuristics.
Killing Science Fiction: Why Conscious States Cannot Be Copied or Repeated

Several philosophical problems arising from the physics of consciousness, including identity, duplication, teleportation, simulation, self-location, and the Boltzmann Brain problem, hinge on one of the most deeply held but unnecessary convictions of physicalism: the assumption that brain states and their corresponding conscious states can in principle be copied. In this paper I will argue against this assumption by attempting to prove the Unique History Theorem, which states, essentially, that conscious correlations to underlying quantum mechanical measurement events must increase with time and that every conscious state uniquely determines its history from an earlier conscious state. By assuming only that consciousness arises from an underlying physical state, I will argue that the physical evolution from a first physical state giving rise to a conscious state to a second physical state giving rise to a later conscious state is unique. Among the consequences of this theorem are that: consciousness is not algorithmic and a conscious state cannot be uploaded to or simulated by a digital computer; a conscious state cannot be copied by duplicating a brain or any other physical state; and a conscious state cannot be repeated or created de novo. These conclusions shed light on the physical nature of consciousness by rendering moot a variety of seemingly paradoxical philosophy and science fiction problems.
Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks

As autonomous vehicles (AVs) need to interact with other road users, it is of importance to comprehensively understand the dynamic traffic environment, especially the future possible trajectories of surrounding vehicles. This paper presents an algorithm for long-horizon trajectory prediction of surrounding vehicles using a dual long short term memory (LSTM) network, which is capable of effectively improving prediction accuracy in strongly interactive driving environments. In contrast to traditional approaches which require trajectory matching and manual feature selection, this method can automatically learn high-level spatial-temporal features of driver behaviors from naturalistic driving data through sequence learning. By employing two blocks of LSTMs, the proposed method feeds the sequential trajectory to the first LSTM for driver intention recognition as an intermediate indicator, which is immediately followed by a second LSTM for future trajectory prediction. Test results from real-world highway driving data show that the proposed method can, in comparison to state-of-art methods, output more accurate and reasonable estimate of different future trajectories over $5s$ time horizon with root mean square error (RMSE) for longitudinal and lateral prediction less than 5.77m and 0.49m, respectively.
Robust Attacks against Multiple Classifiers

We address the challenge of designing optimal adversarial noise algorithms for settings where a learner has access to multiple classifiers. We demonstrate how this problem can be framed as finding strategies at equilibrium in a two-player, zero-sum game between a learner and an adversary. In doing so, we illustrate the need for randomization in adversarial attacks. In order to compute Nash equilibrium, our main technical focus is on the design of best response oracles that can then be implemented within a Multiplicative Weights Update framework to boost deterministic perturbations against a set of models into optimal mixed strategies. We demonstrate the practical effectiveness of our approach on a series of image classification tasks using both linear classifiers and deep neural networks.
From Caesar Cipher to Unsupervised Learning: A New Method for Classifier Parameter Estimation

Many important classification problems, such as object classification, speech recognition, and machine translation, have been tackled by the supervised learning paradigm in the past, where training corpora of parallel input-output pairs are required with high cost. To remove the need for the parallel training corpora has practical significance for real-world applications, and it is one of the main goals of unsupervised learning. Recently, encouraging progress in unsupervised learning for solving such classification problems has been made and the nature of the challenges has been clarified. In this article, we review this progress and disseminate a class of promising new methods to facilitate understanding the methods for machine learning researchers. In particular, we emphasize the key information that enables the success of unsupervised learning - the sequential statistics as the distributional prior in the labels. Exploitation of such sequential statistics makes it possible to estimate parameters of classifiers without the need of paired input-output data. 
In this paper, we first introduce the concept of Caesar Cipher and its decryption, which motivated the construction of the novel loss function for unsupervised learning we use throughout the paper. Then we use a simple but representative binary classification task as an example to derive and describe the unsupervised learning algorithm in a step-by-step, easy-to-understand fashion. We include two cases, one with Bigram language model as the sequential statistics for use in unsupervised parameter estimation, and another with a simpler Unigram language model. For both cases, detailed derivation steps for the learning algorithm are included. Further, a summary table compares computational steps of the two cases in executing the unsupervised learning algorithm for learning binary classifiers.
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

Obstacles hindering the development of capsule networks for challenging NLP applications include poor scalability to large output spaces and less reliable routing processes. In this paper, we introduce: (i) an agreement score to evaluate the performance of routing processes at instance-level; (ii) an adaptive optimizer to enhance the reliability of routing; (iii) capsule compression and partial routing to improve the scalability of capsule networks. We validate our approach on two NLP tasks, namely: multi-label text classification and question answering. Experimental results show that our approach considerably improves over strong competitors on both tasks. In addition, we gain the best results in low-resource settings with few training instances.
Progress and simulations for intranuclear neutron-antineutron transformations in 
Ar1840

With the imminent construction of the Deep Underground Neutrino Experiment (DUNE) and Hyper-Kamiokande, nucleon decay searches as a means to constrain beyond Standard Model (BSM) extensions are once again at the forefront of fundamental physics. Abundant neutrons within these large experimental volumes, along with future high-intensity neutron beams such as the European Spallation Source, offer a powerful, high-precision portal onto this physics through searches for $\mathcal{B}$ and $\mathcal{B}-\mathcal{L}$ violating processes such as neutron--antineutron transformations ($n\rightarrow\bar{n}$), a key prediction of compelling theories of baryogenesis. With this in mind, this paper discusses a novel and self-consistent intranuclear simulation of this process within ${}^{40}_{18} Ar$, which plays the role of both detector and target within DUNE's gigantic liquid argon time projection chambers. An accurate and independent simulation of the resulting intranuclear annihilation respecting important physical correlations and cascade dynamics for this large nucleus is necessary to understand the viability of such rare searches when contrasted against background sources such as atmospheric neutrinos. Recent theoretical improvements to our model, including the first calculation of ${}^{40}_{18} Ar$'s $\bar{n}$-intranuclear suppression factor, and Monte Carlo simulation comparisons to another publicly available $n\rightarrow\bar{n}$ generator within GENIE, are also discussed.