Miracle-dz/UniFAR-FTUs huggingface dataset:
Exploring Context, Attention and Audio Features for Audio Visual Scene-Aware Dialog

We are witnessing a confluence of vision, speech and dialog system technologies that are enabling the IVAs to learn audio-visual groundings of utterances and have conversations with users about the objects, activities and events surrounding them. Recent progress in visual grounding techniques and Audio Understanding are enabling machines to understand shared semantic concepts and listen to the various sensory events in the environment. With audio and visual grounding methods, end-to-end multimodal SDS are trained to meaningfully communicate with us in natural language about the real dynamic audio-visual sensory world around us. In this work, we explore the role of `topics' as the context of the conversation along with multimodal attention into such an end-to-end audio-visual scene-aware dialog system architecture. We also incorporate an end-to-end audio classification ConvNet, AclNet, into our models. We develop and test our approaches on the Audio Visual Scene-Aware Dialog (AVSD) dataset released as a part of the DSTC7. We present the analysis of our experiments and show that some of our model variations outperform the baseline system released for AVSD.
(Mis)Information Operations: An Integrated Perspective

The massive diffusion of social media fosters disintermediation and changes the way users are informed, the way they process reality, and the way they engage in public debate. The cognitive layer of users and the related social dynamics define the nature and the dimension of informational threats. Users show the tendency to interact with information adhering to their preferred narrative and to ignore dissenting information. Confirmation bias seems to account for users decisions about consuming and spreading content; and, at the same time, aggregation of favored information within those communities reinforces group polarization. In this work, the authors address the problem of (mis)information operations with a holistic and integrated approach. Cognitive weakness induced by this new information environment are considered. Moreover, (mis)information operations, with particular reference to the Italian context, are considered; and the fact that the phenomenon is more complex than expected is highlighted. The paper concludes by providing an integrated research roadmap accounting for the possible future technological developments.
Quantile Propagation for Wasserstein-Approximate Gaussian Processes

In this work, we develop a new approximation method to solve the analytically intractable Bayesian inference for Gaussian process models with factorizable Gaussian likelihoods and single-output latent functions. Our method -- dubbed QP -- is similar to the expectation propagation (EP), however it minimizes the $L^2$ Wasserstein distance instead of the Kullback-Leibler (KL) divergence. We consider the specific case in which the non-Gaussian likelihood is approximated by the Gaussian likelihood. We show that QP has the following properties: (1) QP matches quantile functions rather than moments in EP; (2) QP and EP have the same local update for the mean of the approximate Gaussian likelihood; (3) the local variance estimate for the approximate likelihood is smaller for QP than for EP's, addressing EP's over-estimation of the variance; (4) the optimal approximate Gaussian likelihood enjoys a univariate parameterization, reducing memory consumption and computation time. Furthermore, we provide a unified interpretations of EP and QP -- both are coordinate descent algorithms of a KL and an $L^2$ Wasserstein global objective function respectively, under the same assumptions. In the performed experiments, we employ eight real world datasets and we show that QP outperforms EP for the task of Gaussian process binary classification.
Data-Free Adversarial Distillation

Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer. However, almost all existing KD algorithms are data-driven, i.e., relying on a large amount of original training data or alternative data, which is usually unavailable in real-world scenarios. In this paper, we devote ourselves to this challenging problem and propose a novel adversarial distillation mechanism to craft a compact student model without any real-world data. We introduce a model discrepancy to quantificationally measure the difference between student and teacher models and construct an optimizable upper bound. In our work, the student and the teacher jointly act the role of the discriminator to reduce this discrepancy, when a generator adversarially produces some "hard samples" to enlarge it. Extensive experiments demonstrate that the proposed data-free method yields comparable performance to existing data-driven methods. More strikingly, our approach can be directly extended to semantic segmentation, which is more complicated than classification, and our approach achieves state-of-the-art results. Code and pretrained models are available at this https URL.
Low-Complexity Detection for Faster-than-Nyquist Signaling Based on Probabilistic Data Association

In this letter, we investigate the sequence estimation problem of faster-than-Nyquist (FTN) signaling as a promising approach for increasing spectral efficiency (SE) in future communication systems. In doing so, we exploit the concept of Gaussian separability and propose two probabilistic data association (PDA) algorithms with polynomial time complexity to detect binary phase-shift keying (BPSK) FTN signaling. Simulation results show that the proposed PDA algorithm outperforms the recently proposed SSSSE and SSSgb<inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula>SE algorithms for all SE values with a modest increase in complexity. The PDA algorithm approaches the performance of the semidefinite relaxation (SDRSE) algorithm for SE values of 0.96 bits/sec/Hz, and it is within the 0.5 dB signal-to-noise ratio (SNR) penalty at SE values of 1.10 bits/sec/Hz for the fixed values of <inline-formula> <tex-math notation="LaTeX">$\beta = 0.3$ </tex-math></inline-formula>.
Learning temporal structure of the input with a network of integrate-and-fire neurons

The task of the brain is to look for structure in the external input. We study a network of integrate-and-fire neurons with several types of recurrent connections that learns the structure of its time-varying feedforward input by attempting to efficiently represent this input with spikes. The efficiency of the representation arises from incorporating the structure of the input into the decoder, which is implicit in the learned synaptic connectivity of the network. While in the original work of [Boerlin, Machens, Deneve 2013] and [Brendel et al., 2017] the structure learned by the network to make the representation efficient was the low-dimensionality of the feedforward input, in the present work it is its temporal dynamics. The network achieves the efficiency by adjusting its synaptic weights in such a way, that for any neuron in the network, the recurrent input cancels the feedforward for most of the time. We show that if the only temporal structure that the input possesses is that it changes slowly on the time scale of neuronal integration, the dimensionality of the network dynamics is equal to the dimensionality of the input. However, if the input follows a linear differential equation of the first order, the efficiency of the representation can be increased by increasing the dimensionality of the network dynamics in comparison to the dimensionality of the input. If there is only one type of slow synaptic current in the network, the increase is two-fold, while if there are two types of slow synaptic currents that decay with different rates and whose amplitudes can be adjusted separately, it is advantageous to make the increase three-fold. We numerically simulate the network with synaptic weights that imply the most efficient input representation in the above cases. We also propose a learning rule by means of which the corresponding synaptic weights can be learned.
Broad spectral lines in AGNs and supermassive black hole mass measurements

Abstract The mass measurement of supermassive black holes (SMBHs) is a very complex task. Between several methods for SMBH mass measurements, some of them use the spectral lines, which indicate the motion of the emitting/absorbing material around an SMBH. Mostly, there is an assumption of virialization of line emitting gas in the region which is close to the central SMBH. In this paper we will give an overview of methods for the SMBH mass measurements using broad emission spectral lines observed in Type 1 AGNs. First we give the basic idea to use the parameters of broad lines to SMBH mass measurements. After that we give an overview of broad lines from X-ray (Fe kα) to the IR (Pashen and Brecket lines) which have been used for SMBH mass estimates. Additionally, we describe and discuss a new method for SMBH mass measurements using the polarization in the broad lines emitted from Type 1 AGNs.
RECURSION FOR MASUR-VEECH VOLUMES OF MODULI SPACES OF QUADRATIC DIFFERENTIALS

Abstract We derive a quadratic recursion relation for the linear Hodge integrals of the form $\langle \tau _{2}^{n}\lambda _{k}\rangle $ . These numbers are used in a formula for Masur-Veech volumes of moduli spaces of quadratic differentials discovered by Chen, Möller and Sauvaget. Therefore, our recursion provides an efficient way of computing these volumes.
Spectral photon sorting for large-scale Cherenkov and scintillation detectors

We describe here measurements with a new device, the "dichroicon," a Winston-style light concentrator built out of dichroic reflectors, which could allow large-scale neutrino detectors to sort photons by wavelength with small overall light loss. Photon sorting would benefit large-scale water or ice Cherenkov detectors such as Hyper-Kamiokande or ICECUBE by providing a measure of dispersion, which in turn could allow improved position reconstruction and timing. For scintillator detectors like JUNO, upgrades to SNO+ or KamLAND-ZEN, or to water-based liquid scintillator detectors like Theia, dichroicons would provide effective discrimination between Cherenkov and scintillation light, allowing them to operate as true hybrid detectors. Dichroicons are useful for wavelength discrimination in any photon-starved environment in which detection area is limited. We include measurements with a prototype dichroicon using first a Cherenkov source to show spectral photon sorting works as expected. We then present measurements of two different LAB-based liquid scintillator sources, and demonstrate discrimination between Cherenkov and scintillation light. On the benchtop we can identify Cherenkov light with better than 90\% purity while maintaining a high collection efficiency for the scintillation light. First results from simulations of a large-scale detector are also presented.
TentacleNet: A Pseudo-Ensemble Template for Accurate Binary Convolutional Neural Networks

Binarization is an attractive strategy for implementing lightweight Deep Convolutional Neural Networks (CNNs). Despite the unquestionable savings offered, memory footprint above all, it may induce an excessive accuracy loss that prevents a widespread use. This work elaborates on this aspect introducing TentacleNet, a new template designed to improve the predictive performance of binarized CNNs via parallelization. Inspired by the ensemble learning theory, it consists of a compact topology that is end-to-end trainable and organized to minimize memory utilization. Experimental results collected over three realistic benchmarks show TentacleNet fills the gap left by classical binary models, ensuring substantial memory savings w.r. t. state-of-the-art binary ensemble methods.
A Survey of Deep Learning Applications to Autonomous Vehicle Control

Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment. However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalising previously learned rules to new scenarios. For these reasons, the use of deep learning for vehicle control is becoming increasingly popular. Although important advancements have been achieved in this field, these works have not been fully summarised. This paper surveys a wide range of research works reported in the literature which aim to control a vehicle through deep learning methods. Although there exists overlap between control and perception, the focus of this paper is on vehicle control, rather than the wider perception problem which includes tasks such as semantic segmentation and object detection. The paper identifies the strengths and limitations of available deep learning methods through comparative analysis and discusses the research challenges in terms of computation, architecture selection, goal specification, generalisation, verification and validation, as well as safety. Overall, this survey brings timely and topical information to a rapidly evolving field relevant to intelligent transportation systems.
Exceptional topology of non-Hermitian systems

The current understanding of the role of topology in non-Hermitian (NH) systems and its far-reaching physical consequences observable in a range of dissipative settings are reviewed. In particular, how the paramount and genuinely NH concept of exceptional degeneracies, at which both eigenvalues and eigenvectors coalesce, leads to phenomena drastically distinct from the familiar Hermitian realm is discussed. An immediate consequence is the ubiquitous occurrence of nodal NH topological phases with concomitant open Fermi-Seifert surfaces, where conventional band-touching points are replaced by the aforementioned exceptional degeneracies. Furthermore, new notions of gapped phases including topological phases in single-band systems are detailed, and the manner in which a given physical context may affect the symmetry-based topological classification is clarified. A unique property of NH systems with relevance beyond the field of topological phases consists of the anomalous relation between bulk and boundary physics, stemming from the striking sensitivity of NH matrices to boundary conditions. Unifying several complementary insights recently reported in this context, a picture of intriguing phenomena such as the NH bulk-boundary correspondence and the NH skin effect is put together. Finally, applications of NH topology in both classical systems including optical setups with gain and loss, electric circuits, and mechanical systems and genuine quantum systems such as electronic transport settings at material junctions and dissipative cold-atom setups are reviewed.
Emotion Recognition from Speech

In this work, we conduct an extensive comparison of various approaches to speech based emotion recognition systems. The analyses were carried out on audio recordings from Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). After pre-processing the raw audio files, features such as Log-Mel Spectrogram, Mel-Frequency Cepstral Coefficients (MFCCs), pitch and energy were considered. The significance of these features for emotion classification was compared by applying methods such as Long Short Term Memory (LSTM), Convolutional Neural Networks (CNNs), Hidden Markov Models (HMMs) and Deep Neural Networks (DNNs). On the 14-class (2 genders x 7 emotions) classification task, an accuracy of 68% was achieved with a 4-layer 2 dimensional CNN using the Log-Mel Spectrogram features. We also observe that, in emotion recognition, the choice of audio features impacts the results much more than the model complexity.
An improper estimator with optimal excess risk in misspecified density estimation and logistic regression

We introduce a procedure for predictive conditional density estimation under logarithmic loss, which we call SMP (Sample Minmax Predictor). This estimator minimizes a new general excess risk bound for supervised statistical learning. On standard examples, this bound scales as $d/n$ with $d$ the model dimension and $n$ the sample size, and critically remains valid under model misspecification. Being an improper (out-of-model) procedure, SMP improves over within-model estimators such as the maximum likelihood estimator, whose excess risk degrades under misspecification. Compared to approaches reducing to the sequential problem, our bounds remove suboptimal $\log n$ factors, addressing an open problem from Gr\"unwald and Kotlowski for the considered models, and can handle unbounded classes. For the Gaussian linear model, the predictions and risk bound of SMP are governed by leverage scores of covariates, nearly matching the optimal risk in the well-specified case without conditions on the noise variance or approximation error of the linear model. For logistic regression, SMP provides a non-Bayesian approach to calibration of probabilistic predictions relying on virtual samples, and can be computed by solving two logistic regressions. It achieves a non-asymptotic excess risk of $O ( (d + B^2R^2)/n )$, where $R$ bounds the norm of features and $B$ that of the comparison parameter; by contrast, no within-model estimator can achieve better rate than $\min( {B R}/{\sqrt{n}}, {d e^{BR}}/{n} )$ in general. This provides a computationally more efficient alternative to Bayesian approaches, which require approximate posterior sampling, thereby partly answering a question by Foster et al. (2018).
A Catalog of Galaxy Clusters Identified from SCUSS, SDSS, and UNWISE

This paper presents the identification of galaxy clusters from the photometric redshift catalog based on three imaging surveys of SCUSS, Sloan Digital Sky Survey, and unWISE. By applying a fast clustering algorithm, we obtain a total of 19,610 clusters in the redshift range of 0.05 < z < 0.65 over a sky area of about 3700 deg2 in the south Galactic gap. Monte Carlo simulations show that the false detection rate is about 8.9%. The redshift uncertainty is estimated to be about 0.013. The mass and richness of detected clusters are derived through the calibration based on the measurements of X-ray emission and Sunyaev–Zel’dovich effect. The median mass is 1.2 × 1014 M⊙.
A Survey of Deep Reinforcement Learning in Video Games

Deep reinforcement learning (DRL) has made great achievements since proposed. Generally, DRL agents receive high-dimensional inputs at each step, and make actions according to deep-neural-network-based policies. This learning mechanism updates the policy to maximize the return with an end-to-end method. In this paper, we survey the progress of DRL methods, including value-based, policy gradient, and model-based algorithms, and compare their main techniques and properties. Besides, DRL plays an important role in game artificial intelligence (AI). We also take a review of the achievements of DRL in various video games, including classical Arcade games, first-person perspective games and multi-agent real-time strategy games, from 2D to 3D, and from single-agent to multi-agent. A large number of video game AIs with DRL have achieved super-human performance, while there are still some challenges in this domain. Therefore, we also discuss some key points when applying DRL methods to this field, including exploration-exploitation, sample efficiency, generalization and transfer, multi-agent learning, imperfect information, and delayed spare rewards, as well as some research directions.
Learned SVD: solving inverse problems via hybrid autoencoding

Our world is full of physics-driven data where effective mappings between data manifolds are desired. There is an increasing demand for understanding combined model-driven and data-driven learning methods. We propose a nonlinear, learned singular value decomposition (L-SVD), which combines autoencoders that simultaneously learn and connect latent codes for desired signals and given measurements. Classical solution methods for inverse problems are based on regularisation techniques via SVD and variational methods. An open topic in deep learning for inverse problems is how to achieve model reduction via data dimensionality reduction to obtain a regularised inversion. We investigate this topic and provide a promising direction for solving inverse problems in cases where the underlying physics are not fully understood or have very complex behaviour. We show that the building blocks of learned inversion maps can be obtained automatically, with improved performance upon classical methods and better interpretability than black-box methods.
Sum-Product Network Decompilation

There exists a dichotomy between classical probabilistic graphical models, such as Bayesian networks (BNs), and modern tractable models, such as sum-product networks (SPNs). The former have generally intractable inference, but allow a high level of interpretability, while the latter admits a wide range of tractable inference routines, but are typically harder to interpret. Due to this dichotomy, tools to convert between BNs and SPNs are desirable. While one direction -- compiling BNs into SPNs -- is well discussed in Darwiche's seminal work on arithmetic circuit compilation, the converse direction -- decompiling SPNs into BNs -- has received surprisingly little attention. In this paper, we fill this gap by proposing SPN2BN, an algorithm that decompiles an SPN into a BN. SPN2BN has several salient features when compared to the only other two works decompiling SPNs. Most significantly, the BNs returned by SPN2BN are minimal independence-maps. Secondly, SPN2BN is more parsimonious with respect to the introduction of latent variables. Thirdly, the output BN produced by SPN2BN can be precisely characterized with respect to the compiled BN. More specifically, a certain set of directed edges will be added to the input BN, giving what we will call the moral-closure. It immediately follows that there is a set of BNs related to the input BN that will also return the same moral closure. Lastly, it is established that our compilation-decompilation process is idempotent. We confirm our results with systematic experiments on a number of synthetic BNs.
CAN UVB VARIATIONS RECONCILE SIMULATED QUASAR ABSORPTION LINES AT HIGH REDSHIFT?

In this work we present new calculations of the observables associated with synthetic metal and HI absorption lines in the spectra of high redshift quasars, in-spired by questions and limitations raised in work with a uniform Haardt-Madau 2012 ultraviolet background (UVB). We introduce variations at z ≈ 6 to the UVB and HI self–shielding and explore the sensitivity of the absorption features to modifications of the hardness of the UVB. We find that observed SiIV and low ionization states (e.g. CII, SiII, OI) are well represented by a soft UV ionizing field at z = 6, but this same prescription, fails to reproduce the statistical properties of the observed CIV ion absorber population. We conclude that small variations in the UVB (not greater than a dex below Haardt-Madau 2012 emissivity at 1 Ryd) and HI SSh at z ≈ 6 play a major role in improving the estimation of metal ions and HI statistics at high z.