nils-herrmann/clean_scirepeval huggingface dataset:
OmniNet: A unified architecture for multi-modal multi-task learning

Transformer is a popularly used neural network architecture, especially for language understanding. We introduce an extended and unified architecture which can be used for tasks involving a variety of modalities like image, text, videos, etc. We propose a spatio-temporal cache mechanism that enables learning spatial dimension of the input in addition to the hidden states corresponding to the temporal input sequence. The proposed architecture further enables a single model to support tasks with multiple input modalities as well as asynchronous multi-task learning, thus we refer to it as OmniNet. For example, a single instance of OmniNet can concurrently learn to perform the tasks of part-of-speech tagging, image captioning, visual question answering and video activity recognition. We demonstrate that training these four tasks together results in about three times compressed model while retaining the performance in comparison to training them individually. We also show that using this neural network pre-trained on some modalities assists in learning an unseen task. This illustrates the generalization capacity of the self-attention mechanism on the spatio-temporal cache present in OmniNet.
A fully 3D multi-path convolutional neural network with feature fusion and feature weighting for automatic lesion identification in brain MRI images

We propose a fully 3D multi-path convolutional network to predict stroke lesions from 3D brain MRI images. Our multi-path model has independent encoders for different modalities containing residual convolutional blocks, weighted multi-path feature fusion from different modalities, and weighted fusion modules to combine encoder and decoder features. Compared to existing 3D CNNs like DeepMedic, 3D U-Net, and AnatomyNet, our networks achieves the highest statistically significant cross-validation accuracy of 60.5% on the large ATLAS benchmark of 220 patients. We also test our model on multi-modal images from the Kessler Foundation and Medical College Wisconsin and achieve a statistically significant cross-validation accuracy of 65%, significantly outperforming the multi-modal 3D U-Net and DeepMedic. Overall our model offers a principled, extensible multi-path approach that outperforms multi-channel alternatives and achieves high Dice accuracies on existing benchmarks.
Accounting for total variation and robustness in profiling health care providers.

Monitoring outcomes of health care providers, such as patient deaths, hospitalizations, and hospital readmissions, helps in assessing the quality of health care. We consider a large database on patients being treated at dialysis facilities in the United States, and the problem of identifying facilities with outcomes that are better than or worse than expected. Analyses of such data have been commonly based on random or fixed facility effects, which have shortcomings that can lead to unfair assessments. A primary issue is that they do not appropriately account for variation between providers that is outside the providers' control due, for example, to unobserved patient characteristics that vary between providers. In this article, we propose a smoothed empirical null approach that accounts for the total variation and adapts to different provider sizes. The linear model provides an illustration that extends easily to other non-linear models for survival or binary outcomes, for example. The empirical null method is generalized to allow for some variation being due to quality of care. These methods are examined with numerical simulations and applied to the monitoring of survival in the dialysis facility data.
Stability selection enables robust learning of partial differential equations from limited noisy data

We present a statistical learning framework for robust identification of partial differential equations from noisy spatiotemporal data. Extending previous sparse regression approaches for inferring PDE models from simulated data, we address key issues that have thus far limited the application of these methods to noisy experimental data, namely their robustness against noise and the need for manual parameter tuning. We address both points by proposing a stability-based model selection scheme to determine the level of regularization required for reproducible recovery of the underlying PDE. This avoids manual parameter tuning and provides a principled way to improve the method's robustness against noise in the data. Our stability selection approach, termed PDE-STRIDE, can be combined with any sparsity-promoting penalized regression model and provides an interpretable criterion for model component importance. We show that in particular the combination of stability selection with the iterative hard-thresholding algorithm from compressed sensing provides a fast, parameter-free, and robust computational framework for PDE inference that outperforms previous algorithmic approaches with respect to recovery accuracy, amount of data required, and robustness to noise. We illustrate the performance of our approach on a wide range of noise-corrupted simulated benchmark problems, including 1D Burgers, 2D vorticity-transport, and 3D reaction-diffusion problems. We demonstrate the practical applicability of our method on real-world data by considering a purely data-driven re-evaluation of the advective triggering hypothesis for an embryonic polarization system in C.~elegans. Using fluorescence microscopy images of C.~elegans zygotes as input data, our framework is able to recover the PDE model for the regulatory reaction-diffusion-flow network of the associated proteins.
DEGREES OF RANDOMIZED COMPUTABILITY

In this survey we discuss work of Levin and V’yugin on collections of sequences that are non-negligible in the sense that they can be computed by a probabilistic algorithm with positive probability. More precisely, Levin and V’yugin introduced an ordering on collections of sequences that are closed under Turing equivalence. Roughly speaking, given two such collections 
$\mathcal {A}$
 and 
$\mathcal {B}$
 , 
$\mathcal {A}$
 is below 
$\mathcal {B}$
 in this ordering if 
$\mathcal {A}\setminus \mathcal {B}$
 is negligible. The degree structure associated with this ordering, the Levin–V’yugin degrees (or 
$\mathrm {LV}$
 -degrees), can be shown to be a Boolean algebra, and in fact a measure algebra. We demonstrate the interactions of this work with recent results in computability theory and algorithmic randomness: First, we recall the definition of the Levin–V’yugin algebra and identify connections between its properties and classical properties from computability theory. In particular, we apply results on the interactions between notions of randomness and Turing reducibility to establish new facts about specific LV-degrees, such as the LV-degree of the collection of 1-generic sequences, that of the collection of sequences of hyperimmune degree, and those collections corresponding to various notions of effective randomness. Next, we provide a detailed explanation of a complex technique developed by V’yugin that allows the construction of semi-measures into which computability-theoretic properties can be encoded. We provide two examples of the use of this technique by explicating a result of V’yugin’s about the LV-degree of the collection of Martin-Löf random sequences and extending the result to the LV-degree of the collection of sequences of DNC degree.
Unsupervised Task Design to Meta-Train Medical Image Classifiers

Meta-training has been empirically demonstrated to be the most effective pre-training method for few-shot learning of medical image classifiers (i.e., classifiers modeled with small training sets). However, the effectiveness of meta-training relies on the availability of a reasonable number of hand-designed classification tasks, which are costly to obtain, and consequently rarely available. In this paper, we propose a new method to unsupervisedly design a large number of classification tasks to meta-train medical image classifiers. We evaluate our method on a breast dynamically contrast enhanced magnetic resonance imaging (DCE-MRI) data set that has been used to benchmark few-shot training methods of medical image classifiers. Our results show that the proposed unsupervised task design to meta-train medical image classifiers builds a pre-trained model that, after fine-tuning, produces better classification results than other unsupervised and supervised pre-training methods, and competitive results with respect to meta-training that relies on hand-designed classification tasks.
Decoding The Style And Bias of Song Lyrics

The central idea of this paper is to gain a deeper understanding of song lyrics computationally. We focus on two aspects: style and biases of song lyrics. All prior works to understand these two aspects are limited to manual analysis of a small corpus of song lyrics. In contrast, we analyzed more than half a million songs spread over five decades. We characterize the lyrics style in terms of vocabulary, length, repetitiveness, speed, and readability. We have observed that the style of popular songs significantly differs from other songs. We have used distributed representation methods and WEAT test to measure various gender and racial biases in the song lyrics. We have observed that biases in song lyrics correlate with prior results on human subjects. This correlation indicates that song lyrics reflect the biases that exist in society. Increasing consumption of music and the effect of lyrics on human emotions makes this analysis important.
Before and beyond the Wilson-Cowan equations.

The Wilson-Cowan equations represent a landmark in the history of computational neuroscience. Among the insights Wilson and Cowan offered for neuroscience, they crystallized an approach to modeling neural dynamics and brain function. Although their iconic equations are used in various guises today, the ideas that led to their formulation and the relationship to other approaches are not well known. Here, we give a little context to some of the biological and theoretical concepts that lead to the Wilson-Cowan equations and discuss how to extend beyond them.
Comparison of Classical Machine Learning Approaches on Bangla Textual Emotion Analysis

Detecting emotions from text is an extension of simple sentiment polarity detection. Instead of considering only positive or negative sentiments, emotions are conveyed using more tangible manner; thus, they can be expressed as many shades of gray. This paper manifests the results of our experimentation for fine-grained emotion analysis on Bangla text. We gathered and annotated a text corpus consisting of user comments from several Facebook groups regarding socio-economic and political issues, and we made efforts to extract the basic emotions (sadness, happiness, disgust, surprise, fear, anger) conveyed through these comments. Finally, we compared the results of the five most popular classical machine learning techniques namely Naive Bayes, Decision Tree, k-Nearest Neighbor (k-NN), Support Vector Machine (SVM) and K-Means Clustering with several combinations of features. Our best model (SVM with a non-linear radial-basis function (RBF) kernel) achieved an overall average accuracy score of 52.98% and an F1 score (macro) of 0.3324
Searches for Technosignatures: The State of the Profession

The search for life in the universe is a major theme of astronomy and astrophysics for the next decade. Searches for technosignatures are complementary to searches for biosignatures, in that they offer an alternative path to discovery, and address the question of whether complex (i.e. technological) life exists elsewhere in the Galaxy. This approach has been endorsed in prior Decadal Reviews and National Academies reports, and yet the field still receives almost no federal support in the US. Because of this lack of support, searches for technosignatures, precisely the part of the search of greatest public interest, suffers from a very small pool of trained practitioners. A major source of this issue is institutional inertia at NASA, which avoids the topic as a result of decades-past political grandstanding, conflation of the effort with non-scientific topics such as UFOs, and confusion regarding the scope of the term "SETI." The Astro2020 Decadal should address this issue by making developing the field an explicit priority for the next decade. It should recommend that NASA and the NSF support training and curricular development in the field in a way that supports equity and diversity, and make explicit calls for proposals to fund searches for technosignatures.
EEG-Based Emotion Recognition Using Regularized Graph Neural Networks

Electroencephalography (EEG) measures the neuronal activities in different brain regions via electrodes. Many existing studies on EEG-based emotion recognition do not fully exploit the topology of EEG channels. In this paper, we propose a regularized graph neural network (RGNN) for EEG-based emotion recognition. RGNN considers the biological topology among different brain regions to capture both local and global relations among different EEG channels. Specifically, we model the inter-channel relations in EEG signals via an adjacency matrix in a graph neural network where the connection and sparseness of the adjacency matrix are inspired by neuroscience theories of human brain organization. In addition, we propose two regularizers, namely node-wise domain adversarial training (NodeDAT) and emotion-aware distribution learning (EmotionDL), to better handle cross-subject EEG variations and noisy labels, respectively. Extensive experiments on two public datasets, SEED and SEED-IV, demonstrate the superior performance of our model than state-of-the-art models in most experimental settings. Moreover, ablation studies show that the proposed adjacency matrix and two regularizers contribute consistent and significant gain to the performance of our RGNN model. Finally, investigations on the neuronal activities reveal important brain regions and inter-channel relations for EEG-based emotion recognition.
Prioritized Guidance for Efficient Multi-Agent Reinforcement Learning Exploration

Exploration efficiency is a challenging problem in multi-agent reinforcement learning (MARL), as the policy learned by confederate MARL depends on the collaborative approach among multiple agents. Another important problem is the less informative reward restricts the learning speed of MARL compared with the informative label in supervised learning. In this work, we leverage on a novel communication method to guide MARL to accelerate exploration and propose a predictive network to forecast the reward of current state-action pair and use the guidance learned by the predictive network to modify the reward function. An improved prioritized experience replay is employed to better take advantage of the different knowledge learned by different agents which utilizes Time-difference (TD) error more effectively. Experimental results demonstrates that the proposed algorithm outperforms existing methods in cooperative multi-agent environments. We remark that this algorithm can be extended to supervised learning to speed up its training.
Event-Based Feature Extraction Using Adaptive Selection Thresholds

Unsupervised feature extraction algorithms form one of the most important building blocks in machine learning systems. These algorithms are often adapted to the event-based domain to perform online learning in neuromorphic hardware. However, not designed for the purpose, such algorithms typically require significant simplification during implementation to meet hardware constraints, creating trade offs with performance. Furthermore, conventional feature extraction algorithms are not designed to generate useful intermediary signals which are valuable only in the context of neuromorphic hardware limitations. In this work a novel event-based feature extraction method is proposed that focuses on these issues. The algorithm operates via simple adaptive selection thresholds which allow a simpler implementation of network homeostasis than previous works by trading off a small amount of information loss in the form of missed events that fall outside the selection thresholds. The behavior of the selection thresholds and the output of the network as a whole are shown to provide uniquely useful signals indicating network weight convergence without the need to access network weights. A novel heuristic method for network size selection is proposed which makes use of noise events and their feature representations. The use of selection thresholds is shown to produce network activation patterns that predict classification accuracy allowing rapid evaluation and optimization of system parameters without the need to run back-end classifiers. The feature extraction method is tested on both the N-MNIST (Neuromorphic-MNIST) benchmarking dataset and a dataset of airplanes passing through the field of view. Multiple configurations with different classifiers are tested with the results quantifying the resultant performance gains at each processing stage.
Understanding Video Content: Efficient Hero Detection and Recognition for the Game "Honor of Kings"

In order to understand content and automatically extract labels for videos of the game "Honor of Kings", it is necessary to detect and recognize characters (called "hero") together with their camps in the game video. In this paper, we propose an efficient two-stage algorithm to detect and recognize heros in game videos. First, we detect all heros in a video frame based on blood bar template-matching method, and classify them according to their camps (self/ friend/ enemy). Then we recognize the name of each hero using one or more deep convolution neural networks. Our method needs almost no work for labelling training and testing samples in the recognition stage. Experiments show its efficiency and accuracy in the task of hero detection and recognition in game videos.
Behavioural Macroeconomic Policy: New perspectives on time inconsistency

This paper brings together divergent approaches to time inconsistency from macroeconomic policy and behavioural economics. Behavioural discount functions from behavioural microeconomics are embedded into a game-theoretic analysis of temptation versus enforcement to construct an encompassing model, nesting combinations of time consistent and time inconsistent preferences. The analysis presented in this paper shows that, with hyperbolic/quasihyperbolic discounting, the enforceable range of inflation targets is narrowed. This suggests limits to the effectiveness of monetary targets, under certain conditions. The paper concludes with a discussion of monetary policy implications, explored specifically in the light of current macroeconomic policy debates.
Characterization of fracture in topology-optimized bioinspired networks.

Designing strong and robust bioinspired structures requires an understanding of how function arises from the architecture and geometry of materials found in nature. We draw from trabecular bone, a lightweight bone tissue that exhibits a complex, anisotropic microarchitecture, to generate networked structures using multiobjective topology optimization. Starting from an identical volume, we generate multiple different models by varying the objective weights for compliance, surface area, and stability. We examine the relative effects of these objectives on how resultant models respond to simulated mechanical loading and element failure. We adapt a network-based method developed initially in the context of modeling trabecular bone to describe the topology-optimized structures with a graph-theoretical framework, and we use community detection to characterize locations of fracture. This complementary combination of computational methods can provide valuable insights into the strength of bioinspired structures and mechanisms of fracture.
Nucleation of antagonistic organisms and cellular competitions on curved, inflating substrates.

We consider the dynamics of spatially distributed, diffusing populations of organisms with antagonistic interactions. These interactions are found on many length scales, ranging from kilometer-scale animal range dynamics with selection against hybrids to micron-scale interactions between poison-secreting microbial populations. We find that the dynamical line tension at the interface between antagonistic organisms suppresses survival probabilities of small clonal clusters: the line tension introduces a critical cluster size that an organism with a selective advantage must achieve before deterministically spreading through the population. We calculate the survival probability as a function of selective advantage δ and antagonistic interaction strength σ. Unlike a simple Darwinian selective advantage, the survival probability depends strongly on the spatial diffusion constant D_{s} of the strains when σ>0, with suppressed survival when both species are more motile. Finally, we study the survival probability of a single mutant cell at the frontier of a growing spherical cluster of cells, such as the surface of an avascular spherical tumor. Both the inflation and curvature of the frontier significantly enhance the survival probability by changing the critical size of the nucleating cell cluster.
On the Equality of Domination Number and 2-Domination Number

The 2-domination number γ2(G) of a graph G is the minimum cardinality of a set D ⊆ V (G) for which every vertex outside D is adjacent to at least two vertices in D. Clearly, γ2(G) cannot be smaller than the domination number γ(G). We consider a large class of graphs and characterize those members which satisfy γ2 = γ. For the general case, we prove that it is NP-hard to decide whether γ2 = γ holds. We also give a necessary and sufficient condition for a graph to satisfy the equality hereditarily.
Autoencoder-Based Incremental Class Learning without Retraining on Old Data

Incremental class learning, a scenario in continual learning context where classes and their training data are sequentially and disjointedly observed, challenges a problem widely known as catastrophic forgetting. In this work, we propose a novel incremental class learning method that can significantly reduce memory overhead compared to previous approaches. Apart from conventional classification scheme using softmax, our model bases on an autoencoder to extract prototypes for given inputs so that no change in its output unit is required. It stores only the mean of prototypes per class to perform metric-based classification, unlike rehearsal approaches which rely on large memory or generative model. To mitigate catastrophic forgetting, regularization methods are applied on our model when a new task is encountered. We evaluate our method by experimenting on CIFAR-100 and CUB-200-2011 and show that its performance is comparable to the state-of-the-art method with much lower additional memory cost.
Post-Earthquake Assessment of Buildings Using Deep Learning

Classification of the extent of damage suffered by a building in a seismic event is crucial from the safety perspective and repairing work. In this study, authors have proposed a CNN based autonomous damage detection model. Over 1200 images of different types of buildings-1000 for training and 200 for testing classified into 4 categories according to the extent of damage suffered. Categories are namely, no damage, minor damage, major damage, and collapse. Trained network tested by the application of various algorithms with different learning rates. The most optimum results were obtained on the application of VGG16 transfer learning model with a learning rate of 1e-5 as it gave a training accuracy of 97.85% and validation accuracy of up to 89.38%. The model developed has real-time application in the event of an earthquake.