Miracle-dz/UniFAR-FTUs huggingface dataset:
Improved Regret for Zeroth-Order Adversarial Bandit Convex Optimisation

We prove that the information-theoretic upper bound on the minimax regret for zeroth-order adversarial bandit convex optimisation is at most $O(d^{2.5} \sqrt{n} \log(n))$, where $d$ is the dimension and $n$ is the number of interactions. This improves on $O(d^{9.5} \sqrt{n} \log(n)^{7.5}$ by Bubeck et al. (2017). The proof is based on identifying an improved exploratory distribution for convex functions.
Localization and delocalization of ground states of Bose-Einstein condensates under disorder

This paper studies the localization behaviour of Bose-Einstein condensates in disorder potentials, modeled by a Gross-Pitaevskii eigenvalue problem on a bounded interval. In the regime of weak particle interaction, we are able to quantify exponential localization of the ground state, depending on statistical parameters and the strength of the potential. Numerical studies further show delocalization if we leave the identified parameter range, which is in agreement with experimental data. These mathematical and numerical findings allow the prediction of physically relevant regimes where localization of ground states may be observed experimentally.
The recurrent nuclear activity of Fornax A and its interaction with the cold gas

Abstract Sensitive (noise ∼16 μJy beam−1), high-resolution (∼10″) MeerKAT observations of show that its giant lobes have a double-shell morphology, where dense filaments are embedded in a diffuse and extended cocoon, while the central radio jets are confined within the host galaxy. The spectral radio properties of the lobes and jets of reveal that its nuclear activity is rapidly flickering. Multiple episodes of nuclear activity must have formed the radio lobes, for which the last stopped 12 Myr ago. More recently (∼3 Myr ago), a less powerful and short (≲1 Myr) phase of nuclear activity generated the central jets. The distribution and kinematics of the neutral and molecular gas in the centre give insights on the interaction between the recurrent nuclear activity and the surrounding interstellar medium.
Real-Time Face and Landmark Localization for Eyeblink Detection

Pavlovian eyeblink conditioning is a powerful experiment used in the field of neuroscience to measure multiple aspects of how we learn in our daily life. To track the movement of the eyelid during an experiment, researchers have traditionally made use of potentiometers or electromyography. More recently, the use of computer vision and image processing alleviated the need for these techniques but currently employed methods require human intervention and are not fast enough to enable real-time processing. In this work, a face- and landmark-detection algorithm have been carefully combined in order to provide fully automated eyelid tracking, and have further been accelerated to make the first crucial step towards online, closed-loop experiments. Such experiments have not been achieved so far and are expected to offer significant insights in the workings of neurological and psychiatric disorders. Based on an extensive literature search, various different algorithms for face detection and landmark detection have been analyzed and evaluated. Two algorithms were identified as most suitable for eyelid detection: the Histogram-of-Oriented-Gradients (HOG) algorithm for face detection and the Ensemble-of-Regression-Trees (ERT) algorithm for landmark detection. These two algorithms have been accelerated on GPU and CPU, achieving speedups of 1,753$\times$ and 11$\times$, respectively. To demonstrate the usefulness of our eyelid-detection algorithm, a research hypothesis was formed and a well-established neuroscientific experiment was employed: eyeblink detection. Our experimental evaluation reveals an overall application runtime of 0.533 ms per frame, which is 1,101$\times$ faster than the sequential implementation and well within the real-time requirements of eyeblink conditioning in humans, i.e. faster than 500 frames per second.
Efficient Deployment of Conversational Natural Language Interfaces over Databases

Many users communicate with chatbots and AI assistants in order to help them with various tasks. A key component of the assistant is the ability to understand and answer a user’s natural language questions for question-answering (QA). Because data can be usually stored in a structured manner, an essential step involves turning a natural language question into its corresponding query language. However, in order to train most natural language-to-query-language state-of-the-art models, a large amount of training data is needed first. In most domains, this data is not available and collecting such datasets for various domains can be tedious and time-consuming. In this work, we propose a novel method for accelerating the training dataset collection for developing the natural language-to-query-language machine learning models. Our system allows one to generate conversational multi-term data, where multiple turns define a dialogue session, enabling one to better utilize chatbot interfaces. We train two current state-of-the-art NL-to-QL models, on both an SQL and SPARQL-based datasets in order to showcase the adaptability and efficacy of our created data.
A New Accelerated Stochastic Gradient Method with Momentum

In this paper, we propose a novel accelerated stochastic gradient method with momentum, which momentum is the weighted average of previous gradients. The weights decays inverse proportionally with the iteration times. Stochastic gradient descent with momentum (Sgdm) use weights that decays exponentially with the iteration times to generate an momentum term. Using exponentially decaying weights, variants of Sgdm with well designed and complicated formats have been proposed to achieve better performance. The momentum update rules of our method is as simple as that of Sgdm. We provide theoretical convergence properties analyses for our method, which show both the exponentially decay weights and our inverse proportionally decay weights can limit the variance of the moving direction of parameters to be optimized to a region. Experimental results empirically show that our method works well with practical problems and outperforms Sgdm, and it outperforms Adam in convolutional neural networks.
Neural Entity Linking: A Survey of Models based on Deep Learning

This survey presents a comprehensive description of recent neural entity linking (EL) systems developed since 2015 as a result of the “deep learning revolution” in natural language processing. Its goal is to systemize design features of neural entity linking systems and compare their performance to the remarkable classic methods on common benchmarks. This work distills a generic architecture of a neural EL system and discusses its components, such as candidate generation, mention-context encoding, and entity ranking, summarizing prominent methods for each of them. The vast variety of modifications of this general architecture are grouped by several common themes: joint entity mention detection and disambiguation, models for global linking, domain-independent techniques including zero-shot and distant supervision methods, and cross-lingual approaches. Since many neural models take advantage of entity and mention/context embeddings to represent their meaning, this work also overviews prominent entity embedding techniques. Finally, the survey touches on applications of entity linking, focusing on the recently emerged use-case of enhancing deep pre-trained masked language models based on the Transformer architecture.
Sarcasm Detection using Context Separators in Online Discourse

Sarcasm is an intricate form of speech, where meaning is conveyed implicitly. Being a convoluted form of expression, detecting sarcasm is an assiduous problem. The difficulty in recognition of sarcasm has many pitfalls, including misunderstandings in everyday communications, which leads us to an increasing focus on automated sarcasm detection. In the second edition of the Figurative Language Processing (FigLang 2020) workshop, the shared task of sarcasm detection released two datasets, containing responses along with their context sampled from Twitter and Reddit. In this work, we use RoBERTa_{large} to detect sarcasm in both the datasets. We further assert the importance of context in improving the performance of contextual word embedding based models by using three different types of inputs - Response-only, Context-Response, and Context-Response (Separated). We show that our proposed architecture performs competitively for both the datasets. We also show that the addition of a separation token between context and target response results in an improvement of 5.13% in the F1-score in the Reddit dataset.
Data Augmentation for Learning Bilingual Word Embeddings with Unsupervised Machine Translation

Unsupervised bilingual word embedding (BWE) methods learn a linear transformation matrix that maps two monolingual embedding spaces that are separately trained with monolingual corpora. This method assumes that the two embedding spaces are structurally similar, which does not necessarily hold true in general. In this paper, we propose using a pseudo-parallel corpus generated by an unsupervised machine translation model to facilitate structural similarity of the two embedding spaces and improve the quality of BWEs in the mapping method. We show that our approach substantially outperforms baselines and other alternative approaches given the same amount of data, and, through detailed analysis, we argue that data augmentation with the pseudo data from unsupervised machine translation is especially effective for BWEs because (1) the pseudo data makes the source and target corpora (partially) parallel; (2) the pseudo data reflects some nature of the original language that helps learning similar embedding spaces between the source and target languages.
Exemplar-based Generative Facial Editing

Image synthesis has witnessed substantial progress due to the increasing power of generative model. This paper we propose a novel generative approach for exemplar based facial editing in the form of the region inpainting. Our method first masks the facial editing region to eliminates the pixel constraints of the original image, then exemplar based facial editing can be achieved by learning the corresponding information from the reference image to complete the masked region. In additional, we impose the attribute labels constraint to model disentangled encodings in order to avoid undesired information being transferred from the exemplar to the original image editing region. Experimental results demonstrate our method can produce diverse and personalized face editing results and provide far more user control flexibility than nearly all existing methods.
On the Yau‐Tian‐Donaldson Conjecture for Generalized Kähler‐Ricci Soliton Equations

Let (X,D) be a polarized log variety with an effective holomorphic torus action, and Θ be a closed positive torus invariant (1,1) ‐current. For any smooth positive function g defined on the moment polytope of the torus action, we study the Monge‐Ampère equations that correspond to generalized and twisted Kähler‐Ricci g‐solitons. We prove a version of the Yau‐Tian‐Donaldson (YTD) conjecture for these general equations, showing that the existence of solutions is always equivalent to an equivariantly uniform Θ‐twisted g‐Ding‐stability. When Θ is a current associated to a torus invariant linear system, we further show that equivariant special test configurations suffice for testing the stability. Our results allow arbitrary klt singularities and generalize most of previous results on (uniform) YTD conjecture for (twisted) Kähler‐Ricci/Mabuchi solitons or Kähler‐Einstein metrics. © 2022 Wiley Periodicals, Inc.
A Theory of 'Auction as a Search' in Speculative Markets

The tatonnement process in high frequency order driven markets is modeled as a search by buyers for sellers and vice-versa. We propose a total order book model, comprising limit orders and latent orders, in the absence of a market maker. A zero intelligence approach of agents is employed using a diffusion-drift-reaction model, to explain the trading through continuous auctions (price and volume). The search (levy or brownian) for transaction price is the primary diffusion mechanism with other behavioural dynamics in the model inspired from foraging, chemotaxis and robotic search. Analytic and asymptotic analysis is provided for several scenarios and examples. Numerical simulation of the model extends our understanding of the relative performance between brownian, superdiffusive and ballistic search in the model.
privGAN: Protecting GANs from membership inference attacks at low cost to utility

Abstract Generative Adversarial Networks (GANs) have made releasing of synthetic images a viable approach to share data without releasing the original dataset. It has been shown that such synthetic data can be used for a variety of downstream tasks such as training classifiers that would otherwise require the original dataset to be shared. However, recent work has shown that the GAN models and their synthetically generated data can be used to infer the training set membership by an adversary who has access to the entire dataset and some auxiliary information. Current approaches to mitigate this problem (such as DPGAN [1]) lead to dramatically poorer generated sample quality than the original non–private GANs. Here we develop a new GAN architecture (privGAN), where the generator is trained not only to cheat the discriminator but also to defend membership inference attacks. The new mechanism is shown to empirically provide protection against this mode of attack while leading to negligible loss in downstream performances. In addition, our algorithm has been shown to explicitly prevent memorization of the training set, which explains why our protection is so effective. The main contributions of this paper are: i) we propose a novel GAN architecture that can generate synthetic data in a privacy preserving manner with minimal hyperparameter tuning and architecture selection, ii) we provide a theoretical understanding of the optimal solution of the privGAN loss function, iii) we empirically demonstrate the effectiveness of our model against several white and black–box attacks on several benchmark datasets, iv) we empirically demonstrate on three common benchmark datasets that synthetic images generated by privGAN lead to negligible loss in downstream performance when compared against non– private GANs. While we have focused on benchmarking privGAN exclusively on image datasets, the architecture of privGAN is not exclusive to image datasets and can be easily extended to other types of datasets. Repository link: https://github.com/microsoft/privGAN.
Pruning via Iterative Ranking of Sensitivity Statistics

With the introduction of SNIP [arXiv:1810.02340v2], it has been demonstrated that modern neural networks can effectively be pruned before training. Yet, its sensitivity criterion has since been criticized for not propagating training signal properly or even disconnecting layers. As a remedy, GraSP [arXiv:2002.07376v1] was introduced, compromising on simplicity. However, in this work we show that by applying the sensitivity criterion iteratively in smaller steps - still before training - we can improve its performance without difficult implementation. As such, we introduce 'SNIP-it'. We then demonstrate how it can be applied for both structured and unstructured pruning, before and/or during training, therewith achieving state-of-the-art sparsity-performance trade-offs. That is, while already providing the computational benefits of pruning in the training process from the start. Furthermore, we evaluate our methods on robustness to overfitting, disconnection and adversarial attacks as well.
Discovery of High-velocity Hα Emission in the Direction of the Fermi Bubble

Wisconsin H-Alpha Mapper observations reveal high-velocity Hα and [N ii]λ6584 emission lines in the same direction and velocity as ultraviolet absorption-line features that have been previously associated with the biconical gamma-ray lobes known as the Fermi Bubbles. We measure an extinction-corrected intensity of for emission with line center vLSR = −221 ± 3 km s−1, corresponding to an emission measure of . This emission arises at the same velocity as Hubble Space Telescope/Cosmic Origins Spectrograph observations of ultraviolet absorption features detected in the PDS 456 quasar sight line that passes through the northern bubble near l = 10. °4, b = +11. °2. We estimate the total column density of ionized gas in this velocity component to be . The comparison of ionized gas emission and absorption yields an estimate for the characteristic density of ne,c = 1.8 ± 0.6 cm−3 and a characteristic length of Lc = 0.56 ± 0.21 pc assuming 30% solar metallicity. For a temperature of —consistent with the measured line widths and [N ii]/Hα line ratio—the gas has a thermal pressure of . Assuming the gas is ∼6.5 kpc distant, the derived density and pressure appear to be anomalously high for gas ∼1.3 kpc above the Galactic midplane. The large thermal pressure is comparable to both a hot halo or Fermi Bubble model, but suggest that the Hα arises in an overpressurized zone.
Ordinal Non-negative Matrix Factorization for Recommendation

We introduce a new non-negative matrix factorization (NMF) method for ordinal data, called OrdNMF. Ordinal data are categorical data which exhibit a natural ordering between the categories. In particular, they can be found in recommender systems, either with explicit data (such as ratings) or implicit data (such as quantized play counts). OrdNMF is a probabilistic latent factor model that generalizes Bernoulli-Poisson factorization (BePoF) and Poisson factorization (PF) applied to binarized data. Contrary to these methods, OrdNMF circumvents binarization and can exploit a more informative representation of the data. We design an efficient variational algorithm based on a suitable model augmentation and related to variational PF. In particular, our algorithm preserves the scalability of PF and can be applied to huge sparse datasets. We report recommendation experiments on explicit and implicit datasets, and show that OrdNMF outperforms BePoF and PF applied to binarized data.
Web page classification with Google Image Search results

In this paper, we introduce a novel method that combines multiple neural network results to decide the class of the input. In our model, each element is represented by multiple descriptive images. After the training process of the neural network model, each element is classified by calculating its descriptive image results. We apply our idea to the web page classification problem using Google Image Search results as descriptive images. We obtained a classification rate of 94.90% on the WebScreenshots dataset that contains 20000 web sites in 4 classes. The method is easily applicable to similar problems.
Integrating global spatial features in CNN based Hyperspectral/SAR imagery classification

The land cover classification has played an important role in remote sensing because it can intelligently identify things in one huge remote sensing image to reduce the work of humans. However, a lot of classification methods are designed based on the pixel feature or limited spatial feature of the remote sensing image, which limits the classification accuracy and universality of their methods. This paper proposed a novel method to take into the information of remote sensing image, i.e., geographic latitude-longitude information. In addition, a dual-branch convolutional neural network (CNN) classification method is designed in combination with the global information to mine the pixel features of the image. Then, the features of the two neural networks are fused with another fully neural network to realize the classification of remote sensing images. Finally, two remote sensing images are used to verify the effectiveness of our method, including hyperspectral imaging (HSI) and polarimetric synthetic aperture radar (PolSAR) imagery. The result of the proposed method is superior to the traditional single-channel convolutional neural network.
An experimental analysis of the development of the spinal column IV. Morphogenesis of tail vertebrae during regeneration

The formulation of regeneration problems in einbryological terms testifies to the conceptual enrichment of this field by that of experimental embryology. Reciprocally, several fundamental theories of experimental embryology were originally derived f roIn analysis of regencrating systems (Weiss, '25.3, '39 ; Spemann, '38). Nevertheless there are basic differences in ontogenctic and regenerating systems. The varying role of peripheral nerves in their dynamics is one such distinction. Harrison ( '04), Hamburger ( '28)' Fell ( '29) and others report unimpeded dcvelopnient of the embryonic limb bud in the total absence of neural tissue. On the otlier hand Weiss ('25b)' Butler and SchottC ('41), Singer ('52) and others have demonstrated that neural tissue assumes a critical role in the regeneration of amphibian limbs. Dependence of the regenerative processes upon some influence mediated by neural tissue contrasts with the behavior of a cornparable embryonic system. A still sharper divergence bctween an embryoriic and regenerative event is the excitation of lens formation from the cells of the dorsal border of the iris following cxtirpation of the differentiated lens (e.g. Stone and Steiiiitz, ' 5 3 ) . During regeneration the lens is dcrived f rom a tissue quite different from its embryonic source. The regeneration
Two symmetric four-wave mixing signals generated in a medium with anomalous refractive index

We report experimental and theoretical results of two symmetrical signals of degenerate four-wave mixing generated in rubidium vapor. Both nonlinear signals are induced by two almost copropagating laser beams, with k⃗a and k⃗b wave-vectors, and detected simultaneously in the 2k⃗a−k⃗b and 2k⃗b−k⃗a directions. In each direction, we observe a single peak when the two beams are tuned on the closed transition 85Rb 5S 1/2(F = 3) → 5P 3/2(F = 4). The excitation spectra reveal a small frequency separation between the two peaks, which is explained when propagation effects are taken into account. Furthermore, our theoretical analysis shows that a correct description of the frequency position of each peak is achieved with a variable refractive index for both lasers, in which the scanning laser experiences an anomalous window in the refractive index near resonance.
Model-Based Reinforcement Learning with Value-Targeted Regression

This paper studies model-based reinforcement learning (RL) for regret minimization. We focus on finite-horizon episodic RL where the transition model $P$ belongs to a known family of models $\mathcal{P}$, a special case of which is when models in $\mathcal{P}$ take the form of linear mixtures: $P_{\theta} = \sum_{i=1}^{d} \theta_{i}P_{i}$. We propose a model based RL algorithm that is based on optimism principle: In each episode, the set of models that are `consistent' with the data collected is constructed. The criterion of consistency is based on the total squared error of that the model incurs on the task of predicting \emph{values} as determined by the last value estimate along the transitions. The next value function is then chosen by solving the optimistic planning problem with the constructed set of models. We derive a bound on the regret, which, in the special case of linear mixtures, the regret bound takes the form $\tilde{\mathcal{O}}(d\sqrt{H^{3}T})$, where $H$, $T$ and $d$ are the horizon, total number of steps and dimension of $\theta$, respectively. In particular, this regret bound is independent of the total number of states or actions, and is close to a lower bound $\Omega(\sqrt{HdT})$. For a general model family $\mathcal{P}$, the regret bound is derived using the notion of the so-called Eluder dimension proposed by Russo & Van Roy (2014).