Miracle-dz/UniFAR-FTUs huggingface dataset:
Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization

"Forward-only"algorithms, which train neural networks while avoiding a backward pass, have recently gained attention as a way of solving the biologically unrealistic aspects of backpropagation. Here, we first address compelling challenges related to the"forward-only"rules, which include reducing the performance gap with backpropagation and providing an analytical understanding of their dynamics. To this end, we show that the forward-only algorithm with top-down feedback is well-approximated by an"adaptive-feedback-alignment"algorithm, and we analytically track its performance during learning in a prototype high-dimensional setting. Then, we compare different versions of forward-only algorithms, focusing on the Forward-Forward and PEPITA frameworks, and we show that they share the same learning principles. Overall, our work unveils the connections between three key neuro-inspired learning rules, providing a link between"forward-only"algorithms, i.e., Forward-Forward and PEPITA, and an approximation of backpropagation, i.e., Feedback Alignment.
Matching Correlated Inhomogeneous Random Graphs using the k-core Estimator

We consider the task of estimating the latent vertex correspondence between two edge-correlated random graphs with generic, inhomogeneous structure. We study the so-called k-core estimator, which outputs a vertex correspondence that induces a large, common subgraph of both graphs which has minimum degree at least k. We derive sufficient conditions under which the k-core estimator exactly or partially recovers the latent vertex correspondence. Finally, we specialize our general framework to derive new results on exact and partial recovery in correlated stochastic block models, correlated Chung-Lu graphs, and correlated random geometric graphs.
On Achieving Privacy-Preserving State-of-the-Art Edge Intelligence

Deep Neural Network (DNN) Inference in Edge Computing, often called Edge Intelligence, requires solutions to insure that sensitive data confidentiality and intellectual property are not revealed in the process. Privacy-preserving Edge Intelligence is only emerging, despite the growing prevalence of Edge Computing as a context of Machine-Learning-as-a-Service. Solutions are yet to be applied, and possibly adapted, to state-of-the-art DNNs. This position paper provides an original assessment of the compatibility of existing techniques for privacy-preserving DNN Inference with the characteristics of an Edge Computing setup, highlighting the appropriateness of secret sharing in this context. We then address the future role of model compression methods in the research towards secret sharing on DNNs with state-of-the-art performance.
Two-dimensional LMS adaptive filters

A novel two-dimensional least mean square (2-D LMS) algorithm is proposed. This algorithm can converge along both the horizontal and vertical directions of a 2-D plane. In order to compare this algorithm to another 2-D LMS algorithm, both are realized in the form of a 2-D adaptive line enhancer (2-D ALE), and they are applied for to enhancement of images contaminated with noise. The 2-D ALE based on the proposed algorithm does not cause any degradations due to anisotropic processing. >
Solutions of Volterra integral equations with infinite delay

We present several new existence results for a Volterra integral equation with infinite delay. We discuss periodic and bounded solutions. Sufficient conditions for the existence of positive periodic solutions are also provided. The techniques we employ have not been used for this equation before. Our results generalize and complement those in the literature and several examples are presented to show their applicability. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)
Scalability Bottlenecks in Multi-Agent Reinforcement Learning Systems

Multi-Agent Reinforcement Learning (MARL) is a promising area of research that can model and control multiple, autonomous decision-making agents. During online training, MARL algorithms involve performance-intensive computations such as exploration and exploitation phases originating from large observation-action space belonging to multiple agents. In this article, we seek to characterize the scalability bottlenecks in several popular classes of MARL algorithms during their training phases. Our experimental results reveal new insights into the key modules of MARL algorithms that limit the scalability, and outline potential strategies that may help address these performance issues.
Cloud on-demand emulation of quantum dynamics with tensor networks

We introduce a tensor network based emulator, simulating a programmable analog quantum processing unit (QPU). The software package is fully integrated in a cloud platform providing a common interface for executing jobs on a HPC cluster as well as dispatching them to a QPU device. We also present typical emulation use cases in the context of Neutral Atom Quantum Processors, such as evaluating the quality of a state preparation pulse sequence, and solving Maximum Independent Set problems by applying a parallel sweep over a set of input pulse parameter values, for systems composed of a large number of qubits.
2023 The energy cost of local rearrangements, not cooperative eﬀects, makes glasses solid

Which phenomenon slows down the dynamics in super-cooled liquids and turns them into glasses is a long-standing question of condensed-matter [1, 2]. Most popular theories posit that the dynamics becomes cooperative as the temperature decreases: many events must occur in a coordinated fashion on a growing length scale for relaxation to occur [3–5]. Instead, other approaches consider that local barriers associated with the elementary rearrangement of a few particles or ‘excitations’ govern the dynamics [6, 7]. Here we resolve this conundrum by introducing a new algorithm, SEER, that can systematically extract hundreds of excitations and their energy from any given conﬁguration. Remarkably, we ﬁnd at low temperatures that the excitation density of states is essentially shifted to higher energy under cooling. This observation directly predicts how local barriers slow down the dynamics, assuming no cooperative eﬀects. We test this prediction in numerical super-cooled liquids using swap algorithms, allowing us to measure the relaxation time up to milliseconds. The agreement is quantitative, revealing that cooperative eﬀects are absent and leading to new perspectives on the glass transition.
On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: a Riemannian optimization interpretation

In many applications, it is desired to obtain extreme eigenvalues and eigenvectors of large Hermitian matrices by efficient and compact algorithms. In particular, orthogonalization-free methods are preferred for large-scale problems for finding eigenspaces of extreme eigenvalues without explicitly computing orthogonal vectors in each iteration. For the top $p$ eigenvalues, the simplest orthogonalization-free method is to find the best rank-$p$ approximation to a positive semi-definite Hermitian matrix by algorithms solving the unconstrained Burer-Monteiro formulation. We show that the nonlinear conjugate gradient method for the unconstrained Burer-Monteiro formulation is equivalent to a Riemannian conjugate gradient method on a quotient manifold with the Bures-Wasserstein metric, thus its global convergence to a stationary point can be proven. Numerical tests suggest that it is efficient for computing the largest $k$ eigenvalues for large-scale matrices if the largest $k$ eigenvalues are nearly distributed uniformly.
Speech recognition in noise using a projection-based likelihood measure for mixture density HMM's

In this study, a cepstral likelihood measure based on the projection operation is incorporated into a mixture density hidden Markov model (HMM) scheme to improve recognition in the presence of additive noise. The case in which the models are determined only under noise-free conditions is addressed. A background discussion and a derivation of the measure are provided. Recognition experiments are presented showing the usefulness of the proposed measure over the standard Gaussian measure (weighted Euclidean distance) for speaker independent, isolated word recognition in noise. It was found that the proposed mixture weighted projection measure significantly improved performance in several noise types, including white, jittering white, and colored noise. As an example, at an SNR of 10-dB white noise, recognition improved from only 38.4% correct using the Gaussian measure to 83.6% using the developed measure. <<ETX>>
Exploring the Cookieverse: A Multi-Perspective Analysis of Web Cookies

Web cookies have been the subject of many research studies over the last few years. However, most existing research does not consider multiple crucial perspectives that can influence the cookie landscape, such as the client's location, the impact of cookie banner interaction, and from which operating system a website is being visited. In this paper, we conduct a comprehensive measurement study to analyze the cookie landscape for Tranco top-10k websites from different geographic locations and analyze multiple different perspectives. One important factor which influences cookies is the use of cookie banners. We develop a tool, BannerClick, to automatically detect, accept, and reject cookie banners with an accuracy of 99%, 97%, and 87%, respectively. We find banners to be 56% more prevalent when visiting websites from within the EU region. Moreover, we analyze the effect of banner interaction on different types of cookies (i.e., first-party, third-party, and tracking). For instance, we observe that websites send, on average, 5.5x more third-party cookies after clicking ``accept'', underlining that it is critical to interact with banners when performing Web measurements. Additionally, we analyze statistical consistency, evaluate the widespread deployment of consent management platforms, compare landing to inner pages, and assess the impact of visiting a website on a desktop compared to a mobile phone. Our study highlights that all of these factors substantially impact the cookie landscape, and thus a multi-perspective approach should be taken when performing Web measurement studies.
Tasks Decomposition Approaches in Crowdsourcing Software Development

A main characteristic of crowdsourcing software development (CSD) is the complexity of tasks and skills required by workers to achieve successful software crowdsourcing. The tasks proposed to the crowd in CSD are checked to ensure they are manageable and achievable. In general, individual tasks come from general goal-oriented projects. There are practices for breaking down software projects into manageable tasks, known as task decomposition. This study identified task decomposition techniques in software engineering, particularly in the context of CSD. Then, we defined the experienced developers who lead the requester in decomposing the project, preparing tasks, and reviewing submissions. This study explored and addressed decomposition approaches in CSD. Next, we selected projects in TopCoder to identify the task decomposition process in the CSD context. Finally, we concluded with future research directions for investigating decomposition approaches and their effects in the CSD context to ensure successful crowdsourced software projects.
Quantum cosmology, eternal inflation, and swampland conjectures

In light of the recent swampland conjectures, we explore quantum cosmology and eternal inflation beyond the slow roll regime. We consider a model of a closed universe with a scalar field ϕ in the framework of tunneling approach to quantum cosmology. The scalar field potential is assumed to have a maximum at ϕ = 0 and can be approximated in its vicinity as V(ϕ)≈ 3H 2 - 1/2m 2 ϕ 2. Using the instanton method, we find that for m < 2H the dominant nucleation channel for the universe is tunneling to a homogeneous, spherical de Sitter space. For larger values of m/H, the most probable tunneling is to an inhomogeneous closed universe with a domain wall wrapped around its equator. We determine the quantum state of the field ϕ in the nucleated universe by solving the Wheeler-DeWitt equation with tunneling boundary conditions. Our results agree with earlier work which assumed a slow-roll regime m ≪ H. We finally show that spherical universes nucleating with m < 2H undergo stochastic eternal inflation with inflating regions forming a fractal of dimension d > 2. For larger values of m the field ϕ is unstable with respect to formation of domain walls and cannot be described by a perturbative stochastic approach.
CO(4–3) and CO(7–6) maps of the nucleus of NGC 253

Context. Molecular line excitation studies of the nuclei of nearby starburst galaxies yield important information on the starburst phenomena, in particular on the temperature and density of the star-forming gas. Such studies also provide templates for high redshift galaxies with even more extreme star formation. Aims. Fundamental constraints on the physical properties in the nuclear regions of external galaxies can be derived from the spectral energy distribution (i.e., integrated flux density vs. rotational quantum number) of CO rotational emission arising from warm gas. Methods. The resolution and sensitivity of the APEX telescope makes it feasible to perform spatially resolved studies of submillimeter (submm) CO emission from the warm, dense gas in nearby starburst nuclei. Using the FLASH dual-channel heterodyne receiver we mapped emission in the CO J = 4-3 and 7-6 lines toward the archetypical nuclear starburst galaxy NGC 253. Results. Combining our new observations with data from the literature, we derive the CO line SED in the central 250 pc of NGC 253, which peaks near the 6-5 transition and has a shape very similar to that of M 82. All CO transitions in the central region can well be fitted with a single temperature/density Large Velocity Gradient (LVG) model. A good match to the observations is found by assuming kinetic gas temperatures that are comparable to the dust temperature (T kin ≈ 60 K) and a H 2 density of order 10 4 cm -3 . Conclusions. Our very first APEX submm study of a nearby starburst nucleus (NGC 253) meaningfully constrains the physical properties of the star-forming molecular gas it contains. With broader band spectrometers and a chopping secondary coming soon, the impact of APEX on extragalactic astrophysics will be foreseeably significant.
The Monge Gap: A Regularizer to Learn All Transport Maps

Optimal transport (OT) theory has been been used in machine learning to study and characterize maps that can push-forward efficiently a probability measure onto another. Recent works have drawn inspiration from Brenier's theorem, which states that when the ground cost is the squared-Euclidean distance, the ``best'' map to morph a continuous measure in $\mathcal{P}(\Rd)$ into another must be the gradient of a convex function. To exploit that result, [Makkuva+ 2020, Korotin+2020] consider maps $T=\nabla f_\theta$, where $f_\theta$ is an input convex neural network (ICNN), as defined by Amos+2017, and fit $\theta$ with SGD using samples. Despite their mathematical elegance, fitting OT maps with ICNNs raises many challenges, due notably to the many constraints imposed on $\theta$; the need to approximate the conjugate of $f_\theta$; or the limitation that they only work for the squared-Euclidean cost. More generally, we question the relevance of using Brenier's result, which only applies to densities, to constrain the architecture of candidate maps fitted on samples. Motivated by these limitations, we propose a radically different approach to estimating OT maps: Given a cost $c$ and a reference measure $\rho$, we introduce a regularizer, the Monge gap $\mathcal{M}^c_{\rho}(T)$ of a map $T$. That gap quantifies how far a map $T$ deviates from the ideal properties we expect from a $c$-OT map. In practice, we drop all architecture requirements for $T$ and simply minimize a distance (e.g., the Sinkhorn divergence) between $T\sharp\mu$ and $\nu$, regularized by $\mathcal{M}^c_\rho(T)$. We study $\mathcal{M}^c_{\rho}$, and show how our simple pipeline outperforms significantly other baselines in practice.
A Refinement of Sequential Equilibrium

The author proposes a refinement of seq uential equilibrium for extensive form games by generalizing a restri ction proposed for signaling games in Cho and D. M. Kreps (1987). The restriction is that beliefs must not assign positive weight to the p ossibilities that can be excluded through reasonable introspection ba sed on the data available as common knowledge. A new technique is dev eloped in order to prove the existence of forward induction equilibri um, which consists of two steps. First, the author establishes the ge neric existence of forward induction equilibrium by exploiting the re sults of E. Kohlberg and J. F. Mertens (1986). Then, he shows that th e forward induction equilibrium correspondence is upper hemicontinuou s in the outcome space with respect to the changes of parameters of t he game. Copyright 1987 by The Econometric Society.
Low Entropy Communication in Multi-Agent Reinforcement Learning

Communication in multi-agent reinforcement learning has been drawing attention recently for its significant role in cooperation. However, multi-agent systems may suffer from limitations on communication resources and thus need efficient communication techniques in real-world scenarios. According to the Shannon-Hartley theorem, messages to be transmitted reliably in worse channels require lower entropy. Therefore, we aim to reduce message entropy in multi-agent communication. A fundamental challenge is that the gradients of entropy are either 0 or ∞, disabling gradient-based methods. To handle it, we propose a pseudo gradient descent scheme, which reduces entropy by adjusting the distributions of messages wisely. We conduct experiments on two base communication frameworks with six environment settings and find that our scheme can reduce message entropy by up to 90% with nearly no loss of cooperation performance.
Multiphase solutions and their reductions for a nonlocal nonlinear Schrödinger equation with focusing nonlinearity

A nonlocal nonlinear Schrödinger equation with focusing nonlinearity is considered, which has been derived as a continuum limit of the Calogero–Sutherland model in an integrable classical dynamical system. The equation is shown to stem from the compatibility conditions of a system of linear partial differential equations (PDEs), assuring its complete integrability. We construct a nonsingular N‐phase solution (N: positive integer) of the equation by means of a direct method. The features of the one‐ and two‐phase solutions are investigated in comparison with the corresponding solutions of the defocusing version of the equation. We also provide an alternative representation of the N‐phase solution in terms of solutions of a system of nonlinear algebraic equations. Furthermore, the eigenvalue problem associated with the N‐phase solution is discussed briefly with some exact results. Subsequently, we demonstrate that the N‐soliton solution can be obtained simply by taking the long‐wave limit of the N‐phase solution. The similar limiting procedure gives an alternative representation of the N‐soliton solution as well as the exact results related to the corresponding eigenvalue problem.
Interactive Context-Preserving Color Highlighting for Multiclass Scatterplots

Color is one of the main visual channels used for highlighting elements of interest in visualization. However, in multi-class scatterplots, color highlighting often comes at the expense of degraded color discriminability. In this paper, we argue for context-preserving highlighting during the interactive exploration of multi-class scatterplots to achieve desired pop-out effects, while maintaining good perceptual separability among all classes and consistent color mapping schemes under varying points of interest. We do this by first generating two contrastive color mapping schemes with large and small contrasts to the background. Both schemes maintain good perceptual separability among all classes and ensure that when colors from the two palettes are assigned to the same class, they have a high color consistency in color names. We then interactively combine these two schemes to create a dynamic color mapping for highlighting different points of interest. We demonstrate the effectiveness through crowd-sourced experiments and case studies.
Online Real-Time Recurrent Learning Using Sparse Connections and Selective Learning

State construction from sensory observations is an important component of a reinforcement learning agent. One solution for state construction is to use recurrent neural networks. Two popular gradient-based methods for recurrent learning are back-propagation through time (BPTT), and real-time recurrent learning (RTRL). BPTT looks at the complete sequence of observations before computing gradients and is unsuitable for online real-time updates. RTRL can do online updates but scales poorly to large networks. In this paper, we propose two constraints that make RTRL scalable. We show that by either decomposing the network into independent modules or learning a recurrent network incrementally, we can make RTRL scale linearly with the number of parameters. Unlike prior scalable gradient estimation algorithms, such as UORO and Truncated-BPTT, our algorithms do not add noise or bias to the gradient estimate. Instead, they trade oﬀ the functional capacity of the recurrent network to achieve scalable learning. We demonstrate the eﬀectiveness of our approach over Truncated-BPTT on a benchmark inspired by animal learning and in policy evaluation for expert Rainbow-DQN agents in the Arcade Learning Environment (ALE).
Building cross-language corpora for human understanding of privacy policies

Making sure that users understand privacy policies that impact them is a key challenge for a real GDPR deployment. Research studies are mostly carried in English, but in Europe and elsewhere, users speak a language that is not English. Replicating studies in different languages requires the availability of comparable cross-language privacy policies corpora. This work provides a methodology for building comparable cross-language in a national language and a reference study language. We provide an application example of our methodology comparing English and Italian extending the corpus of one of the first studies about users understanding of technical terms in privacy policies. We also investigate other open issues that can make replication harder.
Information-Theoretical Approach to Integrated Pulse-Doppler Radar and Communication Systems

Integrated sensing and communication improves the design of systems by combining sensing and communication functions for increased efficiency, accuracy, and cost savings. The optimal integration requires understanding the trade-off between sensing and communication, but this can be difficult due to the lack of unified performance metrics. In this paper, an information-theoretical approach is used to design the system with a unified metric. A sensing rate is introduced to measure the amount of information obtained by a pulse-Doppler radar system. An approximation and lower bound of the sensing rate is obtained in closed forms. Using both the derived sensing information and communication rates, the optimal bandwidth allocation strategy is found for maximizing the weighted sum of the spectral efficiency for sensing and communication. The simulation results confirm the validity of the approximation and the effectiveness of the proposed bandwidth allocation.