Miracle-dz/UniFAR-FTUs huggingface dataset:
Local dendritic balance enables learning of efficient representations in networks of spiking neurons

Significance Neurons have to represent an enormous amount of sensory information. To represent this information efficiently, neurons have to adapt their connections to the sensory inputs. An unresolved problem is how this learning is possible when neurons fire in a correlated way. Yet, these correlations are ubiquitous in neural spiking, either because sensory input shows correlations or because perfect decorrelation of neural spiking through inhibition fails due to physiological transmission delays. We derived from first principles that neurons can, nonetheless, learn efficient representations if inhibition modulates synaptic plasticity in individual dendritic compartments. Our work questions pairwise Hebbian plasticity as a paradigm for representation learning and draws a link between representation learning and a dendritic balance of input currents. How can neural networks learn to efficiently represent complex and high-dimensional inputs via local plasticity mechanisms? Classical models of representation learning assume that feedforward weights are learned via pairwise Hebbian-like plasticity. Here, we show that pairwise Hebbian-like plasticity works only under unrealistic requirements on neural dynamics and input statistics. To overcome these limitations, we derive from first principles a learning scheme based on voltage-dependent synaptic plasticity rules. Here, recurrent connections learn to locally balance feedforward input in individual dendritic compartments and thereby can modulate synaptic plasticity to learn efficient representations. We demonstrate in simulations that this learning scheme works robustly even for complex high-dimensional inputs and with inhibitory transmission delays, where Hebbian-like plasticity fails. Our results draw a direct connection between dendritic excitatory–inhibitory balance and voltage-dependent synaptic plasticity as observed in vivo and suggest that both are crucial for representation learning.
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification

The experimental landscape in natural language processing for social media is too fragmented. Each year, new shared tasks and datasets are proposed, ranging from classics like sentiment analysis to irony detection or emoji prediction. Therefore, it is unclear what the current state of the art is, as there is no standardized evaluation protocol, neither a strong set of baselines trained on such domain-specific data. In this paper, we propose a new evaluation framework (TweetEval) consisting of seven heterogeneous Twitter-specific classification tasks. We also provide a strong set of baselines as starting point, and compare different language modeling pre-training strategies. Our initial experiments show the effectiveness of starting off with existing pre-trained generic language models, and continue training them on Twitter corpora.
Toward Expressive Singing Voice Correction: On Perceptual Validity of Evaluation Metrics for Vocal Melody Extraction

Singing voice correction (SVC) is an appealing application for amateur singers. Commercial products automate SVC by snapping pitch contours to equal-tempered scales, which could lead to deadpan modifications. Together with the neglect of rhythmic errors, extensive manual corrections are still necessary. In this paper, we present a streamlined system to automate expressive SVC for both pitch and rhythmic errors. Particularly, we extend a previous work by integrating advanced techniques for singing voice separation (SVS) and vocal melody extraction. SVC is achieved by temporally aligning the source-target pair, followed by replacing pitch and rhythm of the source with those of the target. We evaluate the framework by a comparative study for melody extraction which involves both subjective and objective evaluations, whereby we investigate perceptual validity of the standard metrics through the lens of SVC. The results suggest that the high pitch accuracy obtained by the metrics does not signify good perceptual scores.
Automatic Repair of Vulnerable Regular Expressions

A regular expression is called vulnerable if there exist input strings on which the usual backtracking-based matching algorithm runs super linear time. Software containing vulnerable regular expressions are prone to algorithmic-complexity denial of service attack in which the malicious user provides input strings exhibiting the bad behavior. Due to the prevalence of regular expressions in modern software, vulnerable regular expressions are serious threat to software security. While there has been prior work on detecting vulnerable regular expressions, in this paper, we present a first step toward repairing a possibly vulnerable regular expression. Importantly, our method handles real world regular expressions containing extended features such as lookarounds, capturing groups, and backreferencing. (The problem is actually trivial without such extensions since any pure regular expression can be made invulnerable via a DFA conversion.) We build our approach on the recent work on example-based repair of regular expressions by Pan et al. [Pan et al. 2019] which synthesizes a regular expression that is syntactically close to the original one and correctly classifies the given set of positive and negative examples. The key new idea is the use of linear-time constraints, which disambiguate a regular expression and ensure linear time matching. We generate the constraints using an extended nondeterministic finite automaton that supports the extended features in real-world regular expressions. While our method is not guaranteed to produce a semantically equivalent regular expressions, we empirically show that the repaired regular expressions tend to be nearly indistinguishable from the original ones.
Multi-UAV Path Planning for Wireless Data Harvesting With Deep Reinforcement Learning

Harvesting data from distributed Internet of Things (IoT) devices with multiple autonomous unmanned aerial vehicles (UAVs) is a challenging problem requiring flexible path planning methods. We propose a multi-agent reinforcement learning (MARL) approach that, in contrast to previous work, can adapt to profound changes in the scenario parameters defining the data harvesting mission, such as the number of deployed UAVs, number, position and data amount of IoT devices, or the maximum flying time, without the need to perform expensive recomputations or relearn control policies. We formulate the path planning problem for a cooperative, non-communicating, and homogeneous team of UAVs tasked with maximizing collected data from distributed IoT sensor nodes subject to flying time and collision avoidance constraints. The path planning problem is translated into a decentralized partially observable Markov decision process (Dec-POMDP), which we solve through a deep reinforcement learning (DRL) approach, approximating the optimal UAV control policy without prior knowledge of the challenging wireless channel characteristics in dense urban environments. By exploiting a combination of centered global and local map representations of the environment that are fed into convolutional layers of the agents, we show that our proposed network architecture enables the agents to cooperate effectively by carefully dividing the data collection task among themselves, adapt to large complex environments and state spaces, and make movement decisions that balance data collection goals, flight-time efficiency, and navigation constraints. Finally, learning a control policy that generalizes over the scenario parameter space enables us to analyze the influence of individual parameters on collection performance and provide some intuition about system-level benefits.
The impact of noise and topology on opinion dynamics in social networks

We investigate the impact of noise and topology on opinion diversity in social networks. We do so by extending well-established models of opinion dynamics to a stochastic setting where agents are subject both to assimilative forces by their local social interactions, as well as to idiosyncratic factors preventing their population from reaching consensus. We model the latter to account for both scenarios where noise is entirely exogenous to peer influence and cases where it is instead endogenous, arising from the agents’ desire to maintain some uniqueness in their opinions. We derive a general analytical expression for opinion diversity, which holds for any network and depends on the network’s topology through its spectral properties alone. Using this expression, we find that opinion diversity decreases as communities and clusters are broken down. We test our predictions against data describing empirical influence networks between major news outlets and find that incorporating our measure in linear models for the sentiment expressed by such sources on a variety of topics yields a notable improvement in terms of explanatory power.
Finite Continuum-Armed Bandits

We consider a situation where an agent has $T$ ressources to be allocated to a larger number $N$ of actions. Each action can be completed at most once and results in a stochastic reward with unknown mean. The goal of the agent is to maximize her cumulative reward. Non trivial strategies are possible when side information on the actions is available, for example in the form of covariates. Focusing on a nonparametric setting, where the mean reward is an unknown function of a one-dimensional covariate, we propose an optimal strategy for this problem. Under natural assumptions on the reward function, we prove that the optimal regret scales as $O(T^{1/3})$ up to poly-logarithmic factors when the budget $T$ is proportional to the number of actions $N$. When $T$ becomes small compared to $N$, a smooth transition occurs. When the ratio $T/N$ decreases from a constant to $N^{-1/3}$, the regret increases progressively up to the $O(T^{1/2})$ rate encountered in continuum-armed bandits.
SAINT+: Integrating Temporal Features for EdNet Correctness Prediction

We propose SAINT+, a successor of SAINT which is a Transformer based knowledge tracing model that separately processes exercise information and student response information. Following the architecture of SAINT, SAINT+ has an encoder-decoder structure where the encoder applies self-attention layers to a stream of exercise embeddings, and the decoder alternately applies self-attention layers and encoder-decoder attention layers to streams of response embeddings and encoder output. Moreover, SAINT+ incorporates two temporal feature embeddings into the response embeddings: elapsed time, the time taken for a student to answer, and lag time, the time interval between adjacent learning activities. We empirically evaluate the effectiveness of SAINT+ on EdNet, the largest publicly available benchmark dataset in the education domain. Experimental results show that SAINT+ achieves state-of-the-art performance in knowledge tracing with an improvement of 1.25% in area under receiver operating characteristic curve compared to SAINT, the current state-of-the-art model in EdNet dataset.
Perturbed Pricing

We propose a simple randomized rule for the optimization of prices in revenue management with contextual information. It is known that the certainty equivalent pricing rule, albeit popular, is sub-optimal. We show that, by allowing a small amount of randomization around these certainty equivalent prices, the benefits of optimal pricing and low regret are achievable.
Measurement-feedback control of the chiral photon emission from an atom chain into a nanofiber

We theoretically investigate measurement-based feedback control of a laser-driven one-dimensional atomic chain interfaced with a nanofiber. The interfacing leads to all-to-all interactions among the atomic emitters and induces chirality, i.e. the directional emission of photons into a preferred guided mode of the nanofiber. In the setting we consider, the measurement of guided light -- conducted either by photon counting or through homodyne detection of the photocurrent quadratures -- is fed back into the system through a modulation of the driving laser field. We investigate how this feedback scheme influences the photon counting rate and the quadratures of the guided light field. Moreover, we analyse how feedback alters the many-body steady state of the atom chain. Our results provide some insights on how to control and engineer dynamics in light-matter networks realizable with state-of-the-art experimental setups.
Automating Abnormality Detection in Musculoskeletal Radiographs through Deep Learning

This paper introduces MuRAD (Musculoskeletal Radiograph Abnormality Detection tool), a tool that can help radiologists automate the detection of abnormalities in musculoskeletal radiographs (bone X-rays). MuRAD utilizes a Convolutional Neural Network (CNN) that can accurately predict whether a bone X-ray is abnormal, and leverages Class Activation Map (CAM) to localize the abnormality in the image. MuRAD achieves an F1 score of 0.822 and a Cohen's kappa of 0.699, which is comparable to the performance of expert radiologists.
BARThez: a Skilled Pretrained French Sequence-to-Sequence Model

Inductive transfer learning has taken the entire NLP field by storm, with models such as BERT and BART setting new state of the art on countless NLU tasks. However, most of the available models and research have been conducted for English. In this work, we introduce BARThez, the first large-scale pretrained seq2seq model for French. Being based on BART, BARThez is particularly well-suited for generative tasks. We evaluate BARThez on five discriminative tasks from the FLUE benchmark and two generative tasks from a novel summarization dataset, OrangeSum, that we created for this research. We show BARThez to be very competitive with state-of-the-art BERT-based French language models such as CamemBERT and FlauBERT. We also continue the pretraining of a multilingual BART on BARThez’ corpus, and show our resulting model, mBARThez, to significantly boost BARThez’ generative performance.
On the Number of Linear Functions Composing Deep Neural Network: Towards a Refined Definition of Neural Networks Complexity

The classical approach to measure the expressive power of deep neural networks with piecewise linear activations is based on counting their maximum number of linear regions. However, when considering the two different models which are the fully connected and the permutation invariant ones, this measure is unable to distinguish them clearly in term of expressivity. To tackle this, we propose a refined definition of deep neural networks complexity. Instead of counting the number of linear regions directly, we first introduce an equivalence relation between the linear functions composing a DNN and then count those functions relatively to that equivalence relation. We continue with a study of our new complexity measure and verify that it has the good expected properties. It is able to distinguish clearly between the two models mentioned above, it is consistent with the classical measure, and it increases exponentially with depth. That last point confirms the high expressive power of deep networks.
Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures

Existing NLP datasets contain various biases, and models tend to quickly learn those biases, which in turn limits their robustness. Existing approaches to improve robustness against dataset biases mostly focus on changing the training objective so that models learn less from biased examples. Besides, they mostly focus on addressing a specific bias, and while they improve the performance on adversarial evaluation sets of the targeted bias, they may bias the model in other ways, and therefore, hurt the overall robustness. In this paper, we propose to augment the input sentences in the training data with their corresponding predicate-argument structures, which provide a higher-level abstraction over different realizations of the same meaning and help the model to recognize important parts of sentences. We show that without targeting a specific bias, our sentence augmentation improves the robustness of transformer models against multiple biases. In addition, we show that models can still be vulnerable to the lexical overlap bias, even when the training data does not contain this bias, and that the sentence augmentation also improves the robustness in this scenario. We will release our adversarial datasets to evaluate bias in such a scenario as well as our augmentation scripts at this https URL.
Measurement of the transmission secondary electron yield of nanometer-thick films in a prototype Timed Photon Counter

We measure the transmission secondary electron yield of nanometer-thick Al2O3/TiN/Al2O3 films using a prototype version of a Timed Photon Counter (TiPC) . We discuss the method to measure the yield extensively. The yield is then measured as a function of landing energy between 1.2 and 1.8 keV and found to be in the range of 0.1 (1.2 keV) to 0.9 (1.8 keV). These results are in agreement to data obtained by a different, independent method. We therefore conclude that the prototype TiPC is able to characterise the thin films in terms of transmission secondary electron yield. Additionally, observed features which are unrelated to the yield determination are interpreted.
Identifying Similar Movie Characters Quickly but Effectively Using Non-exhaustive Pair-wise Attention

Identifying similar movie characters is a captivating task that can be our first step to understand the commonalities between human characteristics and experiences. Here, we seek to identify similar movie character descriptions and evaluate our findings based on whether they belong to a common fan-curated trope (theme). Rather than simply comparing the embedding representation of character description, we use a pair-wise attention model to make use of complex word/span-level relationships across the two character descriptions to predict the similarity of the two characters. Naively, such a model would require the exhaustive comparison of each character to all other characters, which is an O(n^2) operation with respect to the number of characters, making it unfeasible to be used in practice. We reduced this into an O(n) operation using a two-step approach that involves choosing only a tiny fraction of character-pairs to perform pairwise attention on while still being effective in this task. Our approach performs at least 9-27% better than methods based on state-of-the-art paragraph embedding representations.
Enhanced non-ergodic sub-diffusive regime near the many-body localization transition in the presence of random long-range interactions

We study many-body localization (MBL) in a one-dimensional system of spinless fermions with a deterministic aperiodic potential in the presence of long-range interactions decaying as power-law $V_{ij}/(r_i-r_j)^\alpha$ with distance and having random coefficients $V_{ij}$. We demonstrate that MBL survives even for $\alpha 1/2$. The rich phase diagram obtained here is unique to random nature of long-range interactions. We explain this in terms of the enhanced correlations among local energies of the effective Anderson model induced by random power-law interactions.
FERTILISER APPLICATION PRACTICES AND NUTRIENT DEFICIENCIES IN SMALLHOLDER OIL PALM PLANTATIONS IN INDONESIA

SUMMARY Oil palm has become an important source of revenue for smallholders in Indonesia, but productivity of smallholder plantations is generally poor. Nutrient limitations have been suggested as an important agronomic constraint to yield. Our research aimed to quantify fertiliser use, soil and tissue nutrient status, and palm growth and yield in a sample of independent smallholder plantations. We selected 49 plantations in Indonesia in two provinces with contrasting soils. For all plantations, we obtained self-reported fertiliser use and yield data, collected soil and tissue samples, and analysed vegetative growth. More than 170 kg N ha−1 year−1 was applied in one site, and P was applied in excess of recommended quantities in both sites, but on average farmers applied less than 100 kg K ha−1 year−1. Soils in the palm circle were poor in N, P and K in 29, 40 and 82% of the plantations, and deficiencies were measured in 57, 61 and 80% of the leaflet samples, respectively. We found statistically significant correlations between tissue nutrient concentrations and vegetative growth, but a large part of the variation in the data remained unaccounted for. Single leaf area was reduced in >80% of the plantations. Average yields were estimated to be 50‒70% of the water-limited potential. Our results demonstrate that widespread nutrient imbalances and deficiencies, especially potassium and phosphorus, occur in smallholder oil palm plantations, due to inadequate and unbalanced fertiliser application practices. These deficiencies may be an important underlying cause of the overall poor productivity, which threatens the economic and environmental sustainability of the smallholder sector.
Thresholded LASSO Bandit

In this paper, we revisit sparse stochastic contextual linear bandits. In these problems, feature vectors may be of large dimension $d$, but the reward function depends on a few, say $s_0$, of these features only. We present Thresholded LASSO bandit, an algorithm that (i) estimates the vector defining the reward function as well as its sparse support using the LASSO framework with thresholding, and (ii) selects an arm greedily according to this estimate projected on its support. The algorithm does not require prior knowledge of the sparsity index $s_0$. For this simple algorithm, we establish non-asymptotic regret upper bounds scaling as $\mathcal{O}( \log d + \sqrt{T\log T} )$ in general, and as $\mathcal{O}( \log d + \log T)$ under the so-called margin condition (a setting where arms are well separated). The regret of previous algorithms scales as $\mathcal{O}( \sqrt{T} \log (d T))$ and $\mathcal{O}( \log T \log d)$ in the two settings, respectively. Through numerical experiments, we confirm that our algorithm outperforms existing methods.
Third harmonic generation from collective modes in disordered superconductors

The observation of third-harmonic generation (THG) in superconductors suggests that strong terahertz fields provide a preferential tool to excite collective excitations, e.g., the Higgs mode, which remain elusive in ordinary spectroscopy. From the theoretical point of view, disorder effects must be included. Here, the authors show that by treating disorder nonperturbatively the THG is strongly enhanced by disorder, and phase and charge fluctuations, neglected so far, give a substantial contribution to it. In addition, the computed polarization dependence of the signal within realistic band structures challenges the interpretation of recent experiments in cuprates.