Miracle-dz/UniFAR-FTUs huggingface dataset:
A Review of On-Device Fully Neural End-to-End Automatic Speech Recognition Algorithms

In this paper, we review various end-to-end automatic speech recognition algorithms and their optimization techniques for on-device applications. Conventional speech recognition systems comprise a large number of discrete components such as an acoustic model, a language model, a pronunciation model, a text-normalizer, an inverse-text normalizer, a decoder based on a Weighted Finite State Transducer (WFST), and so on. To obtain sufficiently high speech recognition accuracy with such conventional speech recognition systems, a very large language model (up to 100 GB) is usually needed. Hence, the corresponding WFST size becomes enormous, which prohibits their on-device implementation. Recently, fully neural network end-to-end speech recognition algorithms have been proposed. Examples include speech recognition systems based on Connectionist Temporal Classification (CTC), Recurrent Neural Network Transducer (RNN-T), Attention-based Encoder-Decoder models (AED), Monotonic Chunk-wise Attention (MoChA), transformer-based speech recognition systems, and so on. These fully neural network-based systems require much smaller memory footprints compared to conventional algorithms, therefore their on-device implementation has become feasible. In this paper, we review such end-to-end speech recognition models. We extensively discuss their structures, performance, and advantages compared to conventional algorithms.
Empirical Evaluation of Typical Sparse Fast Fourier Transform Algorithms

Computing the Sparse Fast Fourier Transform(sFFT) has emerged as a critical topic for a long time. The sFFT algorithms decrease the runtime and sampling complexity by taking advantage of the signal’s inherent characteristics that a large number of signals are sparse in the frequency domain. More than ten sFFT algorithms have been proposed, which can be classified into many types according to filter, framework, method of location, method of estimation. In this paper, the technology of these algorithms is completely analyzed in theory. The performance of them is thoroughly tested and verified in practice. The techniques involved in different sFFT algorithms include the following contents: five operations of signal, three methods of frequency bucketization, five methods of location, four methods of estimation, two problems caused by bucketization, three methods to solve these two problems, four algorithmic frameworks. All the above technologies and methods are introduced in detail and examples are given to illustrate the above research. After theoretical research, we make experiments for computing the signals of different SNR, length, sparsity by a standard testing platform and record the run time, percentage of the signal sampled and $L_{0},L_{1},L_{2}$ error with eight different typical sFFT algorithms. The result of experiments satisfies the inferences obtained in theory. It helps us to optimize these algorithms and use them selectively.
Minimum Robust Multi-Submodular Cover for Fairness

In this paper, we study a novel problem, Minimum Robust Multi-Submodular Cover for Fairness (MinRF), as follows: given a ground set V; m monotone submodular functions f_1,. ,f_m; m thresholds T_1,. ,T_m and a non-negative integer r; MinRF asks for the smallest set S such that f_i(S \ X) ≥ T_i for all i ∈ [m] and |X| ≤ r. We prove that MinRF is inapproximable within (1- ε) ln m; and no algorithm, taking fewer than exponential number of queries in term of r, is able to output a feasible set to MinRF with high certainty. Three bicriteria approximation algorithms with performance guarantees are proposed: one for r = 0, one for r = 1, and one for general r. We further investigate our algorithms' performance in two applications of MinRF, Information Propagation for Multiple Groups and Movie Recommendation for Multiple Users. Our algorithms have shown to outperform baseline heuristics in both solution quality and the number of queries in most cases.
Finite-temperature magnetic properties of 
Sm2Fe17Nx
 using an 
ab
 
initio
 effective spin model

In this study, we investigate the finite-temperature magnetic properties of Sm2Fe17Nx (x = 0,3) using an effective spin model constructed based on the information obtained by first-principles calculations. We find that assuming the plausible trivalent Sm3+ configuration results in a model that can satisfactorily describe the magnetization curves of Sm2Fe17N3. By contrast, the model based on the divalent Sm2+ configuration is suitable to reproduce the magnetization curves of Sm2Fe17. These results expand the understanding of how electronic structure affects the magnetic properties of these compounds.
Learning to Check Contract Inconsistencies

Contract consistency is important in ensuring the legal validity of the contract. In many scenarios, a contract is written by filling the blanks in a precompiled form. Due to carelessness, two blanks that should be filled with the same (or different) content may be incorrectly filled with different (or same) content. This will result in the issue of contract inconsistencies, which may severely impair the legal validity of the contract. Traditional methods to address this issue mainly rely on manual contract review, which is labor-intensive and costly. In this work, we formulate a novel Contract Inconsistency Checking (CIC) problem, and design an end-to-end framework, called Pair-wise Blank Resolution (PBR), to solve the CIC problem with high accuracy. Our PBR model contains a novel BlankCoder to address the challenge of modeling meaningless blanks. BlankCoder adopts a two-stage attention mechanism that adequately associates a meaningless blank with its relevant descriptions while avoiding the incorporation of irrelevant context words. Experiments conducted on real-world datasets show the promising performance of our method with a balanced accuracy of 94.05% and an F1 score of 90.90% in the CIC problem.
General Policies, Serializations, and Planning Width

It has been observed that in many of the benchmark planning domains, atomic goals can be reached with a simple polynomial exploration procedure, called IW, that runs in time exponential in the problem width. Such problems have indeed a bounded width: a width that does not grow with the number of problem variables and is often no greater than two. Yet, while the notion of width has become part of the state-of-the-art planning algorithms like BFWS, there is still no good explanation for why so many benchmark domains have bounded width. In this work, we address this question by relating bounded width and serialized width to ideas of generalized planning, where general policies aim to solve multiple instances of a planning problem all at once. We show that bounded width is a property of planning domains that admit optimal general policies in terms of features that are explicitly or implicitly represented in the domain encoding. The results are extended to much larger class of domains with bounded serialized width where the general policies do not have to be optimal. The study leads also to a new simple, meaningful, and expressive language for specifying domain serializations in the form of policy sketches which can be used for encoding domain control knowledge by hand or for learning it from traces. The use of sketches and the meaning of the theoretical results are all illustrated through a number of examples.
On the Stochastic Singular Cucker-Smale Model: Well-Posedness, Collision-Avoidance and Flocking

We study the Cucker-Smale (C-S) flocking systems involving both singularity and noise. We first show the local strong well-posedness for the stochastic singular C-S systems in a general setting, in which the communication weight $\psi$ is merely locally Lipschitz on $(0,\infty)$ and has lower bound (could be negative). Then, for the special case that the singularity at origin is of higher order (i.e. $\alpha\ge1$), we establish the global well-posedness by showing the collision-avoidance in finite time, provided the initial distribution has finite moment. Finally, we will study the large time behavior of the solution when $\psi$ is of zero lower bound, and provide the emergence of conditional flocking and unconditional flocking respectively in stochastic sense.
NAPA: Neural Art Human Pose Amplifier

This is the project report for CSCI-GA.2271-001. We target human pose estimation in artistic images. For this goal, we design an end-to-end system that uses neural style transfer for pose regression. We collect a 277-style set for arbitrary style transfer and build an artistic 281-image test set. We directly run pose regression on the test set and show promising results. For pose regression, we propose a 2d-induced bone map from which pose is lifted. To help such a lifting, we additionally annotate the pseudo 3d labels of the full in-the-wild MPII dataset. Further, we append another style transfer as self supervision to improve 2d. We perform extensive ablation studies to analyze the introduced features. We also compare end-to-end with per-style training and allude to the tradeoff between style transfer and pose regression. Lastly, we generalize our model to the real-world human dataset and show its potentiality as a generic pose model. We explain the theoretical foundation in Appendix. We release code at this https URL, data, and video.
Importance of Signal and Image Processing in Photoacoustic Imaging

Photoacoustic imaging (PAI) is a powerful imaging modality that relies on the PA effect. PAI works on the principle of electromagnetic energy absorption by the exogenous contrast agents and/or endogenous molecules present in the biological tissue, consequently generating ultrasound waves. PAI combines a high optical contrast with a high acoustic spatiotemporal resolution, allowing the non-invasive visualization of absorbers at deep structures. However, due to the optical diffusion and ultrasound attenuation in heterogeneous turbid biological tissue, the quality of the PA images is deteriorated. Therefore, signal and image processing techniques are imperative in PAI to provide high quality images with detailed structural and functional information in deep tissues. Here, we review various signal and image processing techniques that have been developed/implemented in PAI. Our goal is to highlight the importance of image computing in photoacoustic imaging.
Bayesian Optimization - Multi-Armed Bandit Problem

In this report, we survey Bayesian Optimization methods focussed on the Multi-Armed Bandit Problem. We take the help of the paper "Portfolio Allocation for Bayesian Optimization". We report a small literature survey on the acquisition functions and the types of portfolio strategies used in papers discussing Bayesian Optimization. We also replicate the experiments and report our findings and compare them to the results in the paper. Code link: https://colab.research. google.com/drive/1GZ14klEDoe3dcBeZKo5l8qqrKf_GmBDn? usp=sharing#scrollTo=XgIBau3O45_V.
LSTM-based Space Occupancy Prediction towards Efficient Building Energy Management

Energy consumed in buildings takes significant portions of the total global energy usage. A large amount of building energy is used for heating, cooling, ventilation, and air-conditioning (HVAC). However, compared to its importance, building energy management systems nowadays are limited in controlling HVAC based on simple rule-based control (RBC) technologies. The ability to design systems that can efficiently manage HVAC can reduce energy usage and greenhouse gas emissions, and, all in all, it can help us to mitigate climate change. This paper proposes predictive time-series models of occupancy patterns using LSTM. Prediction signal for future room occupancy status on the next time span (e.g., next 30 minutes) can be directly used to operate HVAC. For example, based on the prediction and considering the time for cooling or heating, HVAC can be turned on before the room is being used (e.g., turn on 10 minutes earlier). Also, based on the next room empty prediction timing, HVAC can be turned off earlier, and it can help us increase the efficiency of HVAC while not decreasing comfort. We demonstrate our approach's capabilities using real-world energy data collected from multiple rooms of a university building. We show that LSTM's room occupancy prediction based HVAC control could save energy usage by 50% compared to conventional RBC based control.
A Locational Marginal Pricing Mechanism for Uncertainty Management Based on Improved Multi-Ellipsoidal Uncertainty Set

Large-scale integration of renewable energy sources (RES) brings huge challenges to the power system. A cost-effective reserve deployment and uncertainty pricing mechanism are critical to deal with the uncertainty and variability of RES. To this end, this paper proposes a novel locational marginal pricing mechanism in day-ahead market for managing uncertainties from RES. Firstly, an improved multi-ellipsoidal uncertainty set (IMEUS) considering the temporal correlation and conditional correlation of wind power forecast is formulated to better capture the uncertainty of wind power. The dimension of each ellipsoidal subset is optimized based on a comprehensive evaluation index to reduce the invalid region without large loss of modeling accuracy, so as to reduce the conservatism. Then, an IMEUS-based robust unit commitment (RUC) model and a robust economic dispatch (RED) model are established for the day-ahead market clearing. Both the reserve cost and ramping constraints are considered in the overall dispatch process. Furthermore, based on the Langrangian function of the RED model, a new locational marginal pricing mechanism is developed. The uncertainty locational marginal price (ULMP) is introduced to charge the RES for its uncertainties and reward the generators who provide reserve to mitigate uncertainties. The new pricing mechanism can provide effective price signals to incentivize the uncertainty management in the day-ahead market. Finally, the effectiveness of the proposed methods is verified via numerous simulations on the PJM 5-bus system and IEEE 118-bus system.
Universality in the onset of quantum chaos in many-body systems

We show that the onset of quantum chaos at infinite temperature in two many-body 1D lattice models, the perturbed spin-1/2 XXZ and Anderson models, is characterized by universal behavior. Specifically, we show that the onset of quantum chaos is marked by maxima of the typical fidelity susceptibilities that scale with the square of the inverse average level spacing, saturating their upper bound, and that the strength of the integrability/localization breaking perturbation at these maxima decreases with increasing system size. We also show that the spectral function below the "Thouless" energy (in the quantum chaotic regime) diverges when approaching those maxima. Our results suggest that, in the thermodynamic limit, arbitrarily small integrability/localization breaking perturbations result in quantum chaos in the many-body quantum systems studied here.
TACTO: A Fast, Flexible, and Open-Source Simulator for High-Resolution Vision-Based Tactile Sensors

Simulators perform an important role in prototyping, debugging, and benchmarking new advances in robotics and learning for control. Although many physics engines exist, some aspects of the real world are harder than others to simulate. One of the aspects that have so far eluded accurate simulation is touch sensing. To address this gap, we present TACTO – a fast, flexible, and open-source simulator for vision-based tactile sensors. This simulator allows to render realistic high-resolution touch readings at hundreds of frames per second, and can be easily configured to simulate different vision-based tactile sensors, including DIGIT and OmniTact. In this letter, we detail the principles that drove the implementation of TACTO and how they are reflected in its architecture. We demonstrate TACTO on a perceptual task, by learning to predict grasp stability using touch from 1 million grasps, and on a marble manipulation control task. Moreover, we provide a proof-of-concept that TACTO can be successfully used for Sim2Real applications. We believe that TACTO is a step towards the widespread adoption of touch sensing in robotic applications, and to enable machine learning practitioners interested in multi-modal learning and control.
HeadGAN: Video-and-Audio-Driven Talking Head Synthesis

Recent attempts to solve the problem of talking head synthesis using a single reference image have shown promising results. However, most of them fail to meet the identity preservation problem, or perform poorly in terms of photo-realism, especially in extreme head poses. We propose HeadGAN, a novel reenactment approach that conditions synthesis on 3D face representations, which can be extracted from any driving video and adapted to the facial geometry of any source. We improve the plausibility of mouth movements, by utilising audio features as a complementary input to the Generator. Quantitative and qualitative experiments demonstrate the merits of our approach.
FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction

Facial Expressions induce a variety of high-level details on the 3D face geometry. For example, a smile causes the wrinkling of cheeks or the formation of dimples, while being angry often causes wrinkling of the forehead. Morphable Models (3DMMs) of the human face fail to capture such fine details in their PCA-based representations and consequently cannot generate such details when used to edit expressions. In this work, we introduce FaceDet3D, a first-of-its-kind method that generates - from a single image - geometric facial details that are consistent with any desired target expression. The facial details are represented as a vertex displacement map and used then by a Neural Renderer to photo-realistically render novel images of any single image in any desired expression and view. The Project website is: this http URL
Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation

Holistically understanding an object and its 3D movable parts through visual perception models is essential for enabling an autonomous agent to interact with the world. For autonomous driving, the dynamics and states of vehicle parts such as doors, the trunk, and the bonnet can provide meaningful semantic information and interaction states, which are essential to ensure the safety of the self-driving vehicle. Existing visual perception models mainly focus on coarse parsing such as object bounding box detection or pose estimation and rarely tackle these situations. In this paper, we address this important problem for autonomous driving by solving two critical issues using visual data augmentation. First, to deal with data scarcity, we propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images and then reconstructing human-vehicle interaction scenarios. This allows us to directly edit the real images using the aligned 3D parts, yielding effective training data generation for learning robust deep neural networks (DNNs). Second, to benchmark the quality of 3D part understanding, we collect a large dataset in real world driving scenarios with vehicles in uncommon states (VUS), i.e. with the door or trunk opened, etc. Experiments demonstrate our trained network with visual data augmentation largely outperforms other baselines in terms of 2D detection and instance segmentation accuracy. Our network yields large improvements in discovering and understanding these uncommon cases. Moreover, we plan to release all of the source code, the dataset, and the trained model on GitHub.
Policy Optimization as Online Learning with Mediator Feedback

Policy Optimization (PO) is a widely used approach to address continuous control tasks. In this paper, we introduce the notion of mediator feedback that frames PO as an online learning problem over the policy space. The additional available information, compared to the standard bandit feedback, allows reusing samples generated by one policy to estimate the performance of other policies. Based on this observation, we propose an algorithm, RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST), for regret minimization in PO, that employs a randomized exploration strategy, differently from the existing optimistic approaches. When the policy space is finite, we show that under certain circumstances, it is possible to achieve constant regret, while always enjoying logarithmic regret. We also derive problem-dependent regret lower bounds. Then, we extend RANDOMIST to compact policy spaces. Finally, we provide numerical simulations on finite and compact policy spaces, in comparison with PO and bandit baselines.
Ferroic Hysteresis Modeling

Ferroic hysteresis modeling is an approach applicable to study lagging relationship between ferroic properties and external-dynamic-perturbation behaviors. The understanding to the lagging hysteresis behaviors, such as hysteresis area, coercive field, or remnant magnetization/polarization, is essential to the material-industry development. In this review, fundamental analysis as a microscopic study, including spin models, mean-field method and Monte Carlo simulation, and data processing analysis as a linking bridge between macroscopic analysis and the experiment results, such as the artificial neutral network (ANN) and Fourier analysis, were elaborated on how they can be used to model the dynamic ferroic hysteresis. Then, ferroic behaviors were explained on how they depend on the external perturbations, such as the field parameters and temperature. The responses of ferroic behaviors on these factors, in both qualitatively and quantitatively, were also discussed in details.
Bayes Meets Entailment and Prediction: Commonsense Reasoning with Non-monotonicity, Paraconsistency and Predictive Accuracy

The recent success of Bayesian methods in neuroscience and artificial intelligence gives rise to the hypothesis that the brain is a Bayesian machine. Since logic and learning are both practices of the human brain, it leads to another hypothesis that there is a Bayesian interpretation underlying both logical reasoning and machine learning. In this paper, we introduce a generative model of logical consequence relations. It formalises the process of how the truth value of a sentence is probabilistically generated from the probability distribution over states of the world. We show that the generative model characterises a classical consequence relation, paraconsistent consequence relation and nonmonotonic consequence relation. In particular, the generative model gives a new consequence relation that outperforms them in reasoning with inconsistent knowledge. We also show that the generative model gives a new classification algorithm that outperforms several representative algorithms in predictive accuracy and complexity on the Kaggle Titanic dataset.