Miracle-dz/UniFAR-FTUs huggingface dataset:
A study of latent monotonic attention variants

End-to-end models reach state-of-the-art performance for speech recognition, but global soft attention is not monotonic, which might lead to convergence problems, to instability, to bad generalisation, cannot be used for online streaming, and is also inefficient in calculation. Monotonicity can potentially fix all of this. There are several ad-hoc solutions or heuristics to introduce monotonicity, but a principled introduction is rarely found in literature so far. In this paper, we present a mathematically clean solution to introduce monotonicity, by introducing a new latent variable which represents the audio position or segment boundaries. We compare several monotonic latent models to our global soft attention baseline such as a hard attention model, a local windowed soft attention model, and a segmental soft attention model. We can show that our monotonic models perform as good as the global soft attention model. We perform our experiments on Switchboard 300h. We carefully outline the details of our training and release our code and configs.
MOAI: A methodology for evaluating the impact of indoor airflow in the transmission of COVID-19

Epidemiology models play a key role in understanding and responding to the COVID-19 pandemic. In order to build those models, scientists need to understand contributing factors and their relative importance. A large strand of literature has identified the importance of airflow to mitigate droplets and far-field aerosol transmission risks. However, the specific factors contributing to higher or lower contamination in various settings have not been clearly defined and quantified. As part of the MOAI project (https://moaiapp.com), we are developing a privacy-preserving test and trace app to enable infection cluster investigators to get in touch with patients without having to know their identity. This approach allows involving users in the fight against the pandemic by contributing additional information in the form of anonymous research questionnaires. We first describe how the questionnaire was designed, and the synthetic data was generated based on a review we carried out on the latest available literature. We then present a model to evaluate the risk exposition of a user for a given setting. We finally propose a temporal addition to the model to evaluate the risk exposure over time for a given user.
Video Exploration via Video-Specific Autoencoders

We present simple video-speciﬁc autoencoders that enables human-controllable video exploration. This includes a wide variety of analytic tasks such as (but not limited to) spatial and temporal super-resolution, spatial and temporal editing, object removal, video textures, average video exploration, and correspondence estimation within and across videos. Prior work has independently looked at each of these problems and proposed different formulations. In this work, we observe that a simple autoencoder trained (from scratch) on multiple frames of a speciﬁc video enables one to perform a large variety of video processing and editing tasks. Our tasks are enabled by two key observations: (1) latent codes learned by the autoencoder capture spatial and temporal properties of that video and (2) autoencoders can project out-of-sample inputs onto the video-speciﬁc manifold. For e.g. (1) interpolating latent codes enables temporal super-resolution and user-controllable video textures; (2) manifold reprojection enables spatial super-resolution, object removal, and denoising without training for any of the tasks. Importantly, a two-dimensional visualization of latent codes via principal component analysis acts as a tool for users to both visualize and intuitively control video edits. Finally, we quantitatively contrast our approach with the prior art and found that without any supervision and task-speciﬁc knowledge, our approach can perform comparably to supervised approaches speciﬁcally trained for a task.
OLIVAW: Mastering Othello with neither Humans nor a Penny

We introduce O LIVAW , an AI Othello player adopting the de-sign principles of the famous AlphaGo series. The main motivation behind O LIVAW was to attain exceptional competence in a non-trivial board game, but at a tiny fraction of the cost of its illustrious predecessors. In this paper we show how O LI - VAW successfully met this challenge.
Fixing the Teacher-Student Knowledge Discrepancy in Distillation

Training a small student network with the guidance of a larger teacher network is an effective way to promote the performance of the student. Despite the different types, the guided knowledge used to distill is always kept unchanged for different teacher and student pairs in previous knowledge distillation methods. However, we find that teacher and student models with different networks or trained from different initialization could have distinct feature representations among different channels. (e.g. the high activated channel for different categories). We name this incongruous representation of channels as teacher-student knowledge discrepancy in the distillation process. Ignoring the knowledge discrepancy problem of teacher and student models will make the learning of student from teacher more difficult. To solve this problem, in this paper, we propose a novel student-dependent distillation method, knowledge consistent distillation, which makes teacher's knowledge more consistent with the student and provides the best suitable knowledge to different student networks for distillation. Extensive experiments on different datasets (CIFAR100, ImageNet, COCO) and tasks (image classification, object detection) reveal the widely existing knowledge discrepancy problem between teachers and students and demonstrate the effectiveness of our proposed method. Our method is very flexible that can be easily combined with other state-of-the-art approaches.
High electrochemical performance of 3.9 V LiFeSO4F directly synthesized by a scalable solid- state reaction within 1 h

Fe-based polyanion materials such as LiFePO4 or LiFeBO3 are safe and can achieve high energy density, so they have been considered as cathode materials for large-scale lithium ion batteries. Among these compounds, LiFeSO4F achieves the highest voltage (3.9 V) among Fe2+/Fe3+ redox couples and therefore can have higher energy density than LiFePO4. However, full utilization of 3.9 V LiFeSO4F is severely limited by its non-scalable synthesis process and poor electrochemical activity. Here, we report a method to synthesize 3.9 V LiFeSO4F by a scalable solid-state reaction within 1 h by understanding the thermodynamic stability of 3.9 V LiFeSO4F. The resulting material shows the best electrochemical performance reported to date. The process yields nanosized particles that achieve almost full capacity, 140 mA h g−1, which is 93% of the theoretical capacity and that retain excellent capacity, ∼75 mA h g−1 at 1 C for 150 cycles and high rate capability even at 10 C (6 min). This scalable solid-state reaction for 3.9 V LiFeSO4F makes it a plausible replacement for LiFePO4 in the next generation of lithium ion batteries.
Mid-infrared circumstellar emission of the long-period Cepheid ℓ Carinae resolved with VLTI/MATISSE

Context. The nature of circumstellar envelopes (CSEs) around Cepheids is a matter of ongoing debate. The physical origin of their infrared (IR) excess could be shown to either be made up of a shell of ionized gas, a dust envelope, or a combination of both. Aims. This study is aimed at constraining the geometry and the IR excess of the environment of the bright long-period Cepheid ℓ Car (P = 35.5 days) at mid-IR wavelengths in order to understand its physical nature. Methods. We first used photometric observations in various bands (from the visible domain to the infrared) and Spitzer Space Telescope spectroscopy to constrain the IR excess of ℓ Car. Then we analyzed the VLTI/MATISSE measurements at a specific phase of observation in order to determine the flux contribution as well as the size and shape of the environment of the star in the L band. Finally, we tested the hypothesis of a shell of ionized gas in order to model the IR excess. Results. We report the first detection in the L band of a centro-symmetric extended emission around ℓ Car, of about 1.7 R⋆ in full width at half maximum, producing an excess of about 7.0% in this band.This latter value is used to calibrate the IR excess found when comparing the photometric observations in various bands and quasi-static atmosphere models. In the N band, there is no clear evidence for dust emission from VLTI/MATISSE correlated flux and Spitzer data. On the other side, the modeled shell of ionized gas implies a more compact CSE (1.13 ± 0.02 R⋆) that is also fainter (IR excess of 1% in the L band). Conclusions. We provide new evidence supporting a compact CSE for ℓ Car and we demonstrate the capabilities of VLTI/MATISSE for determining common properties of CSEs. While the compact CSE of ℓ Car is likely to be of a gaseous nature, the tested model of a shell of ionized gas is not able to simultaneously reproduce the IR excess and the interferometric observations. Further Galactic Cepheid observations with VLTI/MATISSE are necessary for determining the properties of CSEs, which may also depend on both the pulsation period and the evolutionary state of the stars.
A Transactive Energy Market Framework Considering Network Constraints and Fairness

The continuous penetration of distributed energy resources (DER) in the electric power grid is driving a new paradigm shift towards transactive energy system (TES), an active and more sustainable system characterized by distributed generation and energy exchanges among consumers and producers in the network. This transition, however, comes with challenges such as dealing with the nonlinear and non-convex power flows of the system, determining an optimal transaction price to maximize overall system welfare, and ensuring fairness for all participants. In this paper, we propose a three-stage transactive energy framework that aims to address these challenges. In the first stage, the cost without trading is calculated which will serve as the reference in the profit maximization problem in the next stage. DER dispatch, power flows and initial transaction payments/incentives of the participants will then be determined in the second stage. A benefit allocation algorithm is applied in the third control stage to determine the optimal transaction price and final payments/incentives that will ensure fairness for trading participants. The proposed framework was tested in an IEEE 33-bus system and results show that fair benefits are given for all participants during trading and the system operates within the network and economic constraints.
Trees, forests, chickens, and eggs: when and why to prune trees in a random forest

Due to their long‐standing reputation as excellent off‐the‐shelf predictors, random forests (RFs) continue to remain a go‐to model of choice for applied statisticians and data scientists. Despite their widespread use, however, until recently, little was known about their inner workings and about which aspects of the procedure were driving their success. Very recently, two competing hypotheses have emerged–one based on interpolation and the other based on regularization. This work argues in favor of the latter by utilizing the regularization framework to reexamine the decades‐old question of whether individual trees in an ensemble ought to be pruned. Despite the fact that default constructions of RFs use near full depth trees in most popular software packages, here we provide strong evidence that tree depth should be seen as a natural form of regularization across the entire procedure. In particular, our work suggests that RFs with shallow trees are advantageous when the signal‐to‐noise ratio in the data is low. In building up this argument, we also critique the newly popular notion of “double descent” in RFs by drawing parallels to U‐statistics and arguing that the noticeable jumps in random forest accuracy are the result of simple averaging rather than interpolation.
Y$^2$-Net FCRN for Acoustic Echo and Noise Suppression

In recent years, deep neural networks (DNNs) were studied as an alternative to traditional acoustic echo cancellation (AEC) algorithms. The proposed models achieved remarkable performance for the separate tasks of AEC and residual echo suppression (RES). A promising network topology is a fully convolutional recurrent network (FCRN) structure, which has already proven its performance on both noise suppression and AEC tasks, individually. However, the combination of AEC, postfiltering, and noise suppression to a single network typically leads to a noticeable decline in the quality of the near-end speech component due to the lack of a separate loss for echo estimation. In this paper, we propose a two-stage model (Y$^2$-Net) which consists of two FCRNs, each with two inputs and one output (Y-Net). The first stage (AEC) yields an echo estimate, which - as a novelty for a DNN AEC model - is further used by the second stage to perform RES and noise suppression. While the subjective listening test of the Interspeech 2021 AEC Challenge mostly yielded results close to the baseline, the proposed method scored an average improvement of 0.46 points over the baseline on the blind testset in double-talk on the instrumental metric DECMOS, provided by the challenge organizers.
Natural Latex Films as Carrier for Casearia sylvestris Swartz Extract Associated with Ciprofloxacin

In order to create a new sustained drug delivery system applied for tissue regeneration, this study beget a natural rubber latex (NRL) film with Casearia sylvestris extract and ciprofloxacin. Manipulated from the biomaterial latex obtained by the rubber tree Hevea brasiliensis , the NRL film has shown to be a great angiogenic compound and also has shown a great potential in being a possible carrier for sustained drug release. In order to increase the therapeutic spectrum of the film two substances was added to it: the C. sylvestris extract which has antiulcer, anti-inflammatory and Original Research Article
Composably-secure data-processing in continuous variable quantum key distribution

Continuous-variable quantum key distribution employs the quadratures of a bosonic mode to establish a secret key between two remote parties. The resulting secret-key rate depends not only on the loss and noise in the communication channel, but also on a series of data-processing steps that are needed for transforming shared correlations into a ﬁnal string of secret bits. Within the general setting of composable ﬁnite-size security, we consider a coherent-state protocol, whose quantum communication is simulated and classical data is post-processed via procedures of parameter estimation, error correction and privacy ampliﬁcation. Such steps are presented in detail and suitably adapted to follow recently-developed tools for composable security analysis. For this advanced approach on data-processing, we provide a Python-based environment that allows us to investigate and optimize the protocol parameters to be used in practical experimental implementations.
A muon-track reconstruction exploiting stochastic losses for large-scale Cherenkov detectors

IceCube is a cubic-kilometer Cherenkov telescope operating at the South Pole. The main goal of IceCube is the detection of astrophysical neutrinos and the identification of their sources. High-energy muon neutrinos are observed via the secondary muons produced in charge current interactions with nuclei in the ice. Currently, the best performing muon track directional reconstruction is based on a maximum likelihood method using the arrival time distribution of Cherenkov photons registered by the experiment's photomultipliers. A known systematic shortcoming of the prevailing method is to assume a continuous energy loss along the muon track. However at energies $>1$ TeV the light yield from muons is dominated by stochastic showers. This paper discusses a generalized ansatz where the expected arrival time distribution is parametrized by a stochastic muon energy loss pattern. This more realistic parametrization of the loss profile leads to an improvement of the muon angular resolution of up to $20\%$ for through-going tracks and up to a factor 2 for starting tracks over existing algorithms. Additionally, the procedure to estimate the directional reconstruction uncertainty has been improved to be more robust against numerical errors.
The nature of synchronization in power systems: a revelation from communication theory

The large-scale integration of converter-interfaced resources in electrical power systems raises new stability threats which call for a new theoretic framework for modelling and analysis. Here we present the theory of power-communication isomorphism to solve this grand challenge. It is revealed that an intrinsic communication mechanism governs the synchronisation of all apparatus in power systems based on which a unified representation for heterogeneous apparatus and behaviours is established. We develop the mathematics to model the dynamic interaction within a power-communication isomorphic system which yield a simple stability criterion for complex systems that can be intuitively interpreted and thus conveniently applied in practice.
Self-Regression Learning for Blind Hyperspectral Image Fusion Without Label

Hyperspectral image fusion (HIF) is critical to a wide range of applications in remote sensing and many computer vision applications. Most traditional HIF methods assume that the observation model is predefined or known. However, in real applications, the observation model involved are often complicated and unknown, which leads to the serious performance drop of many advanced HIF methods. Also, deep learning methods can achieve outstanding performance, but they generally require a large number of image pairs for model training, which are difficult to obtain in realistic scenarios. Towards these issues, we proposed a self-regression learning method that alternatively reconstructs hyperspectral image (HSI) and estimate the observation model. In particular, we adopt an invertible neural network (INN) for restoring the HSI, and two fully-connected network (FCN) for estimating the observation model. Moreover, \emph{SoftMax} nonlinearity is applied to the FCN for satisfying the non-negative, sparsity and equality constraints. Besides, we proposed a local consistency loss function to constrain the observation model by exploring domain specific knowledge. Finally, we proposed an angular loss function to improve spectral reconstruction accuracy. Extensive experiments on both synthetic and real-world dataset show that our model can outperform the state-of-the-art methods
Benchmarks for Deep Off-Policy Evaluation

Off-policy evaluation (OPE) holds the promise of being able to leverage large, offline datasets for both obtaining and selecting complex policies for decision making. The ability to perform evaluation offline is particularly important in many real-world domains, such as healthcare, recommender systems, or robotics, where online data collection is an expensive and potentially dangerous process. Being able to accurately evaluate and select high-performing policies without requiring online interaction could yield significant benefits in safety, time, and cost for these applications. While many OPE methods have been proposed in recent years, comparing results between works is difficult because there is currently a lack of a comprehensive and unified benchmark. Moreover, it is difficult to measure how far algorithms have progressed, due to the lack of challenging evaluation tasks. In order to address this gap, we propose a new benchmark for off-policy evaluation which includes tasks on a range of challenging, high-dimensional control problems, with wide selections of datasets and policies for performing policy selection. The goal of of our benchmark is to provide a standardized measure of progress that is motivated from a set of principles designed to challenge and test the limits of existing OPE methods. We perform a comprehensive evaluation of state-of-the-art algorithms, and we will provide open-source access to all data and code to foster future research in this area.
All-Optical Experimental Control of High-Harmonic Photon Energy

We generate high-order harmonics in a gaseous medium with tunable photon energy using time-domain interferometry of double pulses in a noncollinear generation geometry. The method is based on the fact that the generated harmonics inherit certain spectral properties of the driving laser. The two temporally delayed laser pulses, identical in all parameters, are produced by a custom-made split-and-delay unit utilizing wave-front splitting without a significant energy loss. The arrangement is easy to implement in any attosecond pulse generation beamline, and is suitable for the production of an extreme ultraviolet source with simply and quickly variable central photon energy, useful for a broad range of applications.
Spontaneous valley polarization in two-dimensional organometallic lattices

2D ferrovalley materials that exhibit spontaneous valley polarization are both fundamentally intriguing and practically appealing to be used in valleytronic devices. Usually, the research on 2D ferrovalley materials is mainly focused on inorganic systems, severely suffering from in-plane magnetization. Here, we alternatively show by kp model analysis and high-throughput first-principles calculations that ideal spontaneous valley polarization is present in 2D organometallic lattice. We explore the design principle for organic 2D ferrovalley materials composed of (quasi-)planer molecules and transition-metal atoms in hexagonal lattice, and identify twelve promising candidates. These systems have a ferromagnetic or antiferromagnetic semiconducting state, and importantly they exhibit robust out-of-plane magnetization. The interplay between spin and valley, together with strong spin-orbit coupling of transition-metal atoms, guarantee the spontaneous valley polarization in these systems, facilitating the anomalous valley Hall effect. Our findings significantly broaden the scientific and technological impact of ferrovalley physics.
Perun: Secure Multi-Stakeholder Machine Learning Framework with GPU Support

Confidential multi-stakeholder machine learning (ML) allows multiple parties to perform collaborative data analytics while not revealing their intellectual property, such as ML source code, model, or datasets. State-of-the-art solutions based on homomorphic encryption incur a large performance overhead. Hardware-based solutions, such as trusted execution environments (TEEs), significantly improve the performance in inference computations but still suffer from low performance in training computations, e.g., deep neural networks model training, because of limited availability of protected memory and lack of GPU support. To address this problem, we designed and implemented Perun, a framework for confidential multi-stakeholder machine learning that allows users to make a trade-off between security and performance. Perun executes ML training on hardware accelerators (e.g., GPU) while providing security guarantees using trusted computing technologies, such as trusted platform module and integrity measurement architecture. Less compute-intensive workloads, such as inference, execute only inside TEE, thus at a lower trusted computing base. The evaluation shows that during the ML training on CIFAR-10 and real-world medical datasets, Perun achieved a 161x to 1560x speedup compared to a pure TEE-based approach.