Miracle-dz/UniFAR-FTUs huggingface dataset:
FastSeq: Make Sequence Generation Faster

Transformer-based models have made tremendous impacts in natural language generation. However the inference speed is a bottleneck due to large model size and intensive computing involved in auto-regressive decoding process. We develop FastSeq framework to accelerate sequence generation without accuracy loss. The proposed optimization techniques include an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O. These optimizations are general enough to be applicable to Transformer-based models (e.g., T5, GPT2, and UniLM). Our benchmark results on a set of widely used and diverse models demonstrate 4-9x inference speed gain. Additionally, FastSeq is easy to use with a simple one-line code change. The source code is available at https://github.com/microsoft/fastseq.
Drones for Medical Delivery Considering Different Demands Classes: A Markov Decision Process Approach for Managing Health Centers Dispatching Medical Products

We consider the problem of optimizing the distribution operations of a hub using drones to deliver medical supplies to different geographic regions. Drones are an innovative method with many benefits including low-contact delivery thereby reducing the spread of pandemic and vaccine-preventable diseases. While we focus on medical supply delivery for this work, it is applicable to drone delivery for many other applications, including food, postal items, and e-commerce delivery. In this paper, our goal is to address drone delivery challenges by optimizing the distribution operations at a drone hub that dispatch drones to different geographic locations generating stochastic demands for medical supplies. By considering different geographic locations, we consider different classes of demand that require different flight ranges, which is directly related to the amount of charge held in a drone battery. We classify the stochastic demands based on their distance from the drone hub, use a Markov decision process to model the problem, and perform computational tests using realistic data representing a prominent drone delivery company. We solve the problem using a reinforcement learning method and show its high performance compared with the exact solution found using dynamic programming. Finally, we analyze the results and provide insights for managing the drone hub operations.
Distilling Image Classifiers in Object Detectors

Knowledge distillation constitutes a simple yet effective way to improve the performance of a compact student network by exploiting the knowledge of a more powerful teacher. Nevertheless, the knowledge distillation literature remains limited to the scenario where the student and the teacher tackle the same task. Here, we investigate the problem of transferring knowledge not only across architectures but also across tasks. To this end, we study the case of object detection and, instead of following the standard detector-to-detector distillation approach, introduce a classifier-to-detector knowledge transfer framework. In particular, we propose strategies to exploit the classification teacher to improve both the detector's recognition accuracy and localization performance. Our experiments on several detectors with different backbones demonstrate the effectiveness of our approach, allowing us to outperform the state-of-the-art detector-to-detector distillation methods.
Nullspace Property for Optimality of Minimum Frame Angle Under Invertible Linear Operators

Frames with a large minimum angle between any two distinct frame vectors are desirable in many present day applications. For a unit norm frame, the absolute value of the cosine of the minimum frame angle is also known as coherence. Two frames are equivalent if one can be obtained from the other via left action of an invertible linear operator. Frame angles can change under the action of a linear operator. Most of the existing works solve different optimization problems to find an optimal linear operator that maximizes the minimal frame angle (in other words, minimizes the coherence). In the present work, nevertheless, we consider the question: Is it always possible to find an equivalent frame with smaller coherence for a given frame? In this paper, we derive properties of the initial unit norm frame that can ensure an equivalent frame with strictly larger minimal frame angle compared to the initial one. It turns out that the nullspace property of a certain matrix obtained from the initial frame can guarantee such an equivalent frame. We also present the numerical results that support our theoretical claims.
Continuous and time-discrete non-Markovian system-reservoir interactions: Dissipative coherent quantum feedback in Liouville space

Based on tensor network realizations of path integrals reducing exponential memory scaling to polynomial efficiency and a Liouville space implementation of a time-discrete quantum memory, we investigate a quantum system simultaneously exposed to two structured reservoirs. For this purpose, we employ a numerically exact quasi-2D tensor network combining both diagonal and off-diagonal system-reservoir interactions with a twofold memory for continuous and discrete retardation effects. As a possible example, we study the non-Markovian dynamical interplay between discrete photonic feedback and structured acoustic phonon modes, resulting in emerging inter-reservoir correlations and long-living population trapping within an initially-excited two-level system.
Enhance Convolutional Neural Networks with Noise Incentive Block

As a generic modeling tool, Convolutional Neural Networks (CNNs) have been widely employed in image generation and translation tasks. However, when fed with a flat input, current CNN models may fail to generate vivid results due to the spatially shared convolution kernels. We call it the flatness degradation of CNNs. Unfortunately, such degradation is the greatest obstacles to generate a spatially-variant output from a flat input, which has been barely discussed in the previous literature. To tackle this problem, we propose a model agnostic solution, i.e. Noise Incentive Block (NIB), which serves as a generic plug-in for any CNN generation model. The key idea is to break the flat input condition while keeping the intactness of the original information. Specifically, the NIB perturbs the input data symmetrically with a noise map and reassembles them in the feature domain as driven by the objective function. Extensive experiments show that existing CNN models equipped with NIB survive from the flatness degradation and are able to generate visually better results with richer details in some specific image generation tasks given flat inputs, e.g. semantic image synthesis, data-hidden image generation, and deep neural dithering.
Pattern-defeating Quicksort

A new solution for the Dutch national flag problem is proposed, requiring no three-way comparisons, which gives quicksort a proper worst-case runtime of $O(nk)$ for inputs with $k$ distinct elements. This is used together with other known and novel techniques to construct a hybrid sort that is never significantly slower than regular quicksort while speeding up drastically for many input distributions.
Provably Faster Algorithms for Bilevel Optimization

Bilevel optimization has been widely applied in many important machine learning applications such as hyperparameter optimization and meta-learning. Recently, several momentum-based algorithms have been proposed to solve bilevel optimization problems faster. However, those momentum-based algorithms do not achieve provably better computational complexity than $\mathcal{\widetilde O}(\epsilon^{-2})$ of the SGD-based algorithm. In this paper, we propose two new algorithms for bilevel optimization, where the first algorithm adopts momentum-based recursive iterations, and the second algorithm adopts recursive gradient estimations in nested loops to decrease the variance. We show that both algorithms achieve the complexity of $\mathcal{\widetilde O}(\epsilon^{-1.5})$, which outperforms all existing algorithms by the order of magnitude. Our experiments validate our theoretical results and demonstrate the superior empirical performance of our algorithms in hyperparameter applications.
Casimir-Polder shift of ground-state hyperfine Zeeman sublevels of hydrogen isotopes in a micron-sized metallic cavity at finite temperature

The frequencies of transitions between hyperfine levels of ground-state atoms can be measured with exquisite precision using magnetic-resonance techniques. This makes hyperfine transitions ideal probes of QED effects originating from the interaction of atoms with the quantized electromagnetic field. One of the most remarkable effects predicted by QED is the Casimir-Polder shift experienced by the energy levels of atoms placed near one or more dielectric objects. Here we compute the Casimir-Polder shift and the width of hyperfine transitions between ground-state Zeeman sub-levels of an hydrogen atom placed in a micron-sized metallic cavity, over a range of temperatures extending from cryogenic temperatures to room temperature. Results are presented also for deuterium and tritium. We predict shifts of the hyperfine transitions frequencies of a few tens of Hz that might be measurable with present-day magnetic resonance apparatus.
AugVic: Exploiting BiText Vicinity for Low-Resource NMT

The success of Neural Machine Translation (NMT) largely depends on the availability of large bitext training corpora. Due to the lack of such large corpora in low-resource language pairs, NMT systems often exhibit poor performance. Extra relevant monolingual data often helps, but acquiring it could be quite expensive, especially for low-resource languages. Moreover, domain mismatch between bitext (train/test) and monolingual data might degrade the performance. To alleviate such issues, we propose AUGVIC, a novel data augmentation framework for low-resource NMT which exploits the vicinal samples of the given bitext without using any extra monolingual data explicitly. It can diversify the in-domain bitext data with finer level control. Through extensive experiments on four low-resource language pairs comprising data from different domains, we have shown that our method is comparable to the traditional back-translation that uses extra in-domain monolingual data. When we combine the synthetic parallel data generated from AUGVIC with the ones from the extra monolingual data, we achieve further improvements. We show that AUGVIC helps to attenuate the discrepancies between relevant and distant-domain monolingual data in traditional back-translation. To understand the contributions of different components of AUGVIC, we perform an in-depth framework analysis.
Polynomial magic! Hermite polynomials for private data generation

Kernel mean embedding is a useful tool to compare probability measures. Despite its usefulness, kernel mean embedding considers inﬁnite-dimensional features, which are challenging to handle in the context of differentially private data generation. A recent work [13] proposes to approximate the kernel mean embedding of data distribution using ﬁnite-dimensional random features , where the sensitivity of the features becomes analytically tractable. More importantly, this approach significantly reduces the privacy cost, compared to other known privatization methods (e.g., DP-SGD), as the approximate kernel mean embedding of the data distribution is privatized only once and can then be repeatedly used during training of a generator without incurring any further privacy cost. However, the required number of random features is excessively high, often ten thousand to a hundred thousand, which worsens the sensitivity of the approximate kernel mean embedding. To improve the sensitivity, we propose to replace random features with Hermite polynomial features. Unlike the random features, the Hermite polynomial features are ordered , where the features at the low orders contain more information on the distribution than those at the high orders. Hence, a relatively low order of Hermite polynomial features can more accurately approximate the mean embedding of the data distribution compared to a signiﬁcantly higher number of random features. As a result, using the Hermite polynomial features, we signiﬁcantly improve the privacy-accuracy trade-off, reﬂected in the high quality and diversity of the generated data, when tested on several heterogeneous tabular datasets, as well as several image benchmark datasets.
Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning

Recent theoretical work studies sample-efficient reinforcement learning (RL) extensively in two settings: learning interactively in the environment (online RL), or learning from an offline dataset (offline RL). However, existing algorithms and theories for learning near-optimal policies in these two settings are rather different and disconnected. Towards bridging this gap, this paper initiates the theoretical study of policy finetuning, that is, online RL where the learner has additional access to a"reference policy"$\mu$ close to the optimal policy $\pi_\star$ in a certain sense. We consider the policy finetuning problem in episodic Markov Decision Processes (MDPs) with $S$ states, $A$ actions, and horizon length $H$. We first design a sharp offline reduction algorithm -- which simply executes $\mu$ and runs offline policy optimization on the collected dataset -- that finds an $\varepsilon$ near-optimal policy within $\widetilde{O}(H^3SC^\star/\varepsilon^2)$ episodes, where $C^\star$ is the single-policy concentrability coefficient between $\mu$ and $\pi_\star$. This offline result is the first that matches the sample complexity lower bound in this setting, and resolves a recent open question in offline RL. We then establish an $\Omega(H^3S\min\{C^\star, A\}/\varepsilon^2)$ sample complexity lower bound for any policy finetuning algorithm, including those that can adaptively explore the environment. This implies that -- perhaps surprisingly -- the optimal policy finetuning algorithm is either offline reduction or a purely online RL algorithm that does not use $\mu$. Finally, we design a new hybrid offline/online algorithm for policy finetuning that achieves better sample complexity than both vanilla offline reduction and purely online RL algorithms, in a relaxed setting where $\mu$ only satisfies concentrability partially up to a certain time step.
Landauer Versus Nernst: What is the True Cost of Cooling a Quantum System?

Thermodynamics connects our knowledge of the world to our capability to manipulate and thus to control it. This crucial role of control is exemplified by the third law of thermodynamics, Nernst's unattainability principle, which states that infinite resources are required to cool a system to absolute zero temperature. But what are these resources and how should they be utilized? And how does this relate to Landauer's principle that famously connects information and thermodynamics? We answer these questions by providing a framework for identifying the resources that enable the creation of pure quantum states. We show that perfect cooling is possible with Landauer energy cost given infinite time or control complexity. However, such optimal protocols require complex unitaries generated by an external work source. Restricting to unitaries that can be run solely via a heat engine, we derive a novel Carnot-Landauer limit, along with protocols for its saturation. This generalizes Landauer's principle to a fully thermodynamic setting, leading to a unification with the third law and emphasizes the importance of control in quantum thermodynamics.
A general approach for Explanations in terms of Middle Level Features

Nowadays, it is growing interest to make Machine Learning (ML) systems more understandable and trusting to general users. Thus, generating explanations for ML system behaviours that are understandable to human beings is a central scientiﬁc and technological issue addressed by the rapidly growing research area of eXplainable Artiﬁcial Intelligence (XAI). Recently, it is becoming more and more evident that new directions to create better explanations should take into account what a good explanation is to a human user, and consequently, develop XAI solutions able to provide user-centred explanations. This paper suggests taking advantage of developing an XAI general approach that allows producing explanations for an ML system behaviour in terms of diﬀerent and user-selected input features, i.e., explanations composed of input properties that the human user can select according to his background knowledge and goals. To this end, we propose an XAI general approach which is able: 1) to construct explanations in terms of input features that represent more salient and un-derstandable input properties for a user, which we call here Middle-Level input Features (MLFs), 2) to be applied to diﬀerent types of MLFs. We experimentally tested our approach on two diﬀerent datasets and using three diﬀerent types of MLFs. The results seem encouraging.
DravidianMultiModality: A Dataset for Multi-modal Sentiment Analysis in Tamil and Malayalam

Human communication is inherently multimodal and asynchronous. Analyzing human emotions and sentiment is an emerging field of artificial intelligence. We are witnessing an increasing amount of multimodal content in local languages on social media about products and other topics. However, there are not many multimodal resources available for under-resourced Dravidian languages. Our study aims to create a multimodal sentiment analysis dataset for the under-resourced Tamil and Malayalam languages. First, we downloaded product or movies review videos from YouTube for Tamil and Malayalam. Next, we created captions for the videos with the help of annotators. Then we labelled the videos for sentiment, and verified the inter-annotator agreement using Fleiss's Kappa. This is the first multimodal sentiment analysis dataset for Tamil and Malayalam by volunteer annotators.
Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos

Distilling analytical models from data has the potential to advance our understanding and prediction of nonlinear dynamics. Although discovery of governing equations based on observed system states (e.g., trajectory time series) has revealed success in a wide range of nonlinear dynamics, uncovering the closed-form equations directly from raw videos still remains an open challenge. To this end, we introduce a novel end-to-end unsupervised deep learning framework to uncover the mathematical structure of equations that governs the dynamics of moving objects in videos. Such an architecture consists of (1) an encoder-decoder network that learns low-dimensional spatial/pixel coordinates of the moving object, (2) a learnable Spatial-Physical Transformation component that creates mapping between the extracted spatial/pixel coordinates and the latent physical states of dynamics, and (3) a numerical integrator-based sparse regression module that uncovers the parsimonious closed-form governing equations of learned physical states and, meanwhile, serves as a constraint to the autoencoder. The efficacy of the proposed method is demonstrated by uncovering the governing equations of a variety of nonlinear dynamical systems depicted by moving objects in videos. The resulting computational framework enables discovery of parsimonious interpretable model in a flexible and accessible sensing environment where only videos are available.
Optical Networks for Composable Data Centers

Composable data centers (DCs) have been proposed to enable greater efficiencies as the uptake of on-demand computing services grows. In this article we give an overview of composable DCs by discussing their enabling technologies, benefits, challenges, and research directions. We then describe a network for composable DCs that leverages optical communication technologies and components to implement a targeted design. Relative to the implementation of a generic design that requires a (high capacity) dedicated transceiver on each point-to-point link on a mesh optical fabric in a composable DC rack, the targeted design can significantly reduce capital expenditure (by up to 34 times) because fewer transceivers are used. This is achieved with little or no degradation of expected performance in composable DCs.
Multiple Kernel Representation Learning on Networks

Learning representations of nodes in a low dimensional space is a crucial task with numerous interesting applications in network analysis, including link prediction, node classification, and visualization. Two popular approaches for this problem are matrix factorization and random walk-based models. In this paper, we aim to bring together the best of both worlds, towards learning node representations. In particular, we propose a weighted matrix factorization model that encodes random walk-based information about nodes of the network. The benefit of this novel formulation is that it enables us to utilize kernel functions without realizing the exact proximity matrix so that it enhances the expressiveness of existing matrix decomposition methods with kernels and alleviates their computational complexities. We extend the approach with a multiple kernel learning formulation that provides the flexibility of learning the kernel as the linear combination of a dictionary of kernels in data-driven fashion. We perform an empirical evaluation on real-world networks, showing that the proposed model outperforms baseline node embedding algorithms in downstream machine learning tasks.
Calculating the Higgs mass in string theory

In this paper, we establish a fully string-theoretic framework for calculating one-loop Higgs masses directly from first principles in perturbative closed string theories. Our framework makes no assumptions other than worldsheet modular invariance and is therefore applicable to all closed strings, regardless of the specific string construction utilized. This framework can also be employed even when spacetime supersymmetry is broken (and even when this breaking occurs at the Planck scale), and can be utilized for all scalar Higgs fields, regardless of the particular gauge symmetries they break. This therefore includes the Higgs field responsible for electroweak symmetry breaking in the Standard Model. Notably, using our framework, we demonstrate that a gravitational modular anomaly generically relates the Higgs mass to the one-loop cosmological constant, thereby yielding a string-theoretic connection between the two fundamental quantities which are known to suffer from hierarchy problems in the absence of spacetime supersymmetry. We also discuss a number of crucial issues involving the use and interpretation of regulators in UV/IR-mixed theories such as string theory, and the manner in which one can extract an EFT description from such theories. Finally, we analyze the running of the Higgs mass within such an EFT description, and uncover the existence of a"dual IR"region which emerges at high energies as the consequence of an intriguing scale-inversion duality symmetry. We also identify a generic stringy effective potential for the Higgs fields in such theories. Our results can therefore serve as the launching point for a rigorous investigation of gauge hierarchy problems in string theory.