Miracle-dz/UniFAR-FTUs huggingface dataset:
Event-based Video Reconstruction via Potential-assisted Spiking Neural Network

Neuromorphic vision sensor is a new bio-inspired imaging paradigm that reports asynchronous, continuously perpixel brightness changes called ‘events’ with high temporal resolution and high dynamic range. So far, the event-based image reconstruction methods are based on artificial neural networks (ANN) or hand-crafted spatiotemporal smoothing techniques. In this paper, we first implement the image reconstruction work via deep spiking neural network (SNN) architecture. As the bio-inspired neural networks, SNNs operating with asynchronous binary spikes distributed over time, can potentially lead to greater computational efficiency on event-driven hardware. We propose a novel Event-based Video reconstruction framework based on a fully Spiking Neural Network (EVSNN), which utilizes Leaky-Integrate-and-Fire (LIF) neuron and Membrane Potential (MP) neuron. We find that the spiking neurons have the potential to store useful temporal information (memory) to complete such time-dependent tasks. Further-more, to better utilize the temporal information, we propose a hybrid potential-assisted framework (PAEVSNN) using the membrane potential of spiking neuron. The proposed neuron is referred as Adaptive Membrane Potential (AMP) neuron, which adaptively updates the membrane potential according to the input spikes. The experimental results demonstrate that our models achieve comparable performance to ANN-based models on IJRR, MVSEC, and HQF datasets. The energy consumptions of EVSNN and PAEVSNN are $19.36\times$ and $7.75\times$ more computationally ef-ficient than their ANN architectures, respectively. The code and pretrained model are available at https://sites.google. com/view/evsnn.
The impact of COVID-19 pandemic on ridesourcing services differed between small towns and large cities

To curb the spread of the ongoing 2019 novel coronavirus (COVID-19), authorities have adopted several non-pharmaceutical (NPIs) and pharmaceutical interventions, which significantly affected our daily activities and mobility patterns. However, it is still unclear how severity of NPIs, COVID-19-related variables, and vaccination rates have affected demand for ridesourcing services, and whether these effects vary across small towns and large cities. We analyzed over 220 million ride requests in the City of Chicago (population: 2.7 million), Illinois, and 52 thousand in the Town of Innisfil (population: 37 thousand), Ontario, to investigate the impact of the COVID-19 pandemic on the ridesourcing demand in the two locations. Overall, the pandemic resulted in fewer trips in areas with higher proportions of seniors and more trips to parks and green spaces. Ridesourcing demand was adversely affected by the stringency index and COVID-19-related variables, and positively affected by vaccination rates. However, compared to Innisfil, ridesourcing services in Chicago experienced higher reductions in demand, were more affected by the number of hospitalizations and deaths, were less impacted by vaccination rates, and had lower recovery rates.
Speeding up Heterogeneous Federated Learning with Sequentially Trained Superclients

Federated Learning (FL) allows training machine learning models in privacy-constrained scenarios by enabling the cooperation of edge devices without requiring local data sharing. This approach raises several challenges due to the different statistical distribution of the local datasets and the clients’ computational heterogeneity. In particular, the presence of highly non-i.i. d. data severely impairs both the performance of the trained neural network and its convergence rate, increasing the number of communication rounds requested to reach a performance comparable to that of the centralized scenario. As a solution, we propose FedSeq, a novel framework leveraging the sequential training of subgroups of heterogeneous clients, i.e. superclients, to emulate the centralized paradigm in a privacy-compliant way. Given a fixed budget of communication rounds, we show that FedSeq outperforms or match several state-of-the-art federated algorithms in terms of final performance and speed of convergence. Finally, our method can be easily integrated with other approaches available in the literature. Empirical results show that combining existing algorithms with FedSeq further improves its final performance and convergence speed. We test our method on CIFAR-10 and CIFAR-100 and prove its effectiveness in both i.i. d. and non-i.i. d. scenarios.1
The art of building a smooth cosmic distance ladder in a perturbed universe

How does a smooth cosmic distance ladder emerge from observations made from a single location in a lumpy Universe? Distances to the Type Ia supernova (SN1A) in the Hubble flow are anchored on local distance measurements to sources that are very nearby. We described how this configuration could be built in a perturbed universe where lumpiness is described as small perturbations on top of a flat Friedmann-Lemaıtre Robertson-Walker (FLRW) spacetime. We show that there is a non-negligible modification (about 11%) to the background FLRW area distance due to the presence of inhomogeneities in the immediate neighbourhood of an observer. We find that the modification is sourced by the electric part of the Weyl tensor indicating a tidal deformation of the local spacetime of the observer. We show in detail how it could impact the calibration of the SN1A absolute magnitude in the Hubble flow. We show that it could potentially resolve the SN1A absolute magnitude and Hubble tensions simultaneously without the need for early or late dark energy.
The ABBE Corpus: Animate Beings Being Emotional

Emotion detection is an established NLP task of demonstrated utility for text understanding. However, basic emotion detection leaves out key information, namely, who is experiencing the emotion in question. For example, it may be the author, the narrator, or a character; or the emotion may correspond to something the audience is supposed to feel, or even be unattributable to a specific being, e.g., when emotions are being discussed per se. We provide the ABBE corpus -- Animate Beings Being Emotional -- a new double-annotated corpus of texts that captures this key information for one class of emotion experiencer, namely, animate beings in the world described by the text. Such a corpus is useful for developing systems that seek to model or understand this specific type of expressed emotion. Our corpus contains 30 chapters, comprising 134,513 words, drawn from the Corpus of English Novels, and contains 2,010 unique emotion expressions attributable to 2,227 animate beings. The emotion expressions are categorized according to Plutchik's 8-category emotion model, and the overall inter-annotator agreement for the annotations was 0.83 Cohen's Kappa, indicating excellent agreement. We describe in detail our annotation scheme and procedure, and also release the corpus for use by other researchers.
Mott-Driven BEC-BCS Crossover in a Doped Spin Liquid Candidate 
κ−(BEDT−TTF)4Hg2.89Br8

The pairing of interacting fermions leading to superfluidity has two limiting regimes: the Bardeen-Cooper-Schrieffer (BCS) scheme for weakly interacting degenerate fermions and the BoseEinstein condensation (BEC) of bosonic pairs of strongly interacting fermions. While the superconductivity that emerges in most metallic systems is the BCS-like electron pairing, strongly correlated electrons with poor Fermi liquidity can condense into the unconventional BEC-like pairs. Quantum spin liquids harbor extraordinary spin correlation free from order and the superconductivity that possibly emerges by carrier doping of the spin liquids is expected to have a peculiar pairing nature. The present study experimentally explores the nature of the pairing condensate in a doped spin-liquid candidate material and under varying pressure, which changes the electron-electron Coulombic interactions across the Mott critical value in the system. The transport measurements reveal that the superconductivity at low pressures is a BEC-like condensate from a non-Fermi liquid and crosses over to a BCS-like condensate from a Fermi liquid at high pressures. The Nernst-effect measurements distinctively illustrate the two regimes of the pairing in terms of its robustness to the magnetic field. The present Mott tuning of the BEC-BCS crossover can be compared to the Feshbach tuning of the BEC-BCS crossover of fermionic cold atoms.
Server-Side Stepsizes and Sampling Without Replacement Provably Help in Federated Optimization

We present a theoretical study of server-side optimization in federated learning. Our results are the first to show that the widely popular heuristic of scaling the client updates with an extra parameter is very useful in the context of Federated Averaging (FedAvg) with local passes over the client data. Each local pass is performed without replacement using Random Reshuffling, which is a key reason we can show improved complexities. In particular, we prove that whenever the local stepsizes are small, and the update direction is given by FedAvg in conjunction with Random Reshuffling over all clients, one can take a big leap in the obtained direction and improve rates for convex, strongly convex, and non-convex objectives. In particular, in non-convex regime we get an enhancement of the rate of convergence from O (ε−3) to O (ε−2). This result is new even for Random Reshuffling performed on a single node. In contrast, if the local stepsizes are large, we prove that the noise of client sampling can be controlled by using a small server-side stepsize. To the best of our knowledge, this is the first time that local steps provably help to overcome the communication bottleneck. Together, our results on the advantage of large and small server-side stepsizes give a formal justification for the practice of adaptive server-side optimization in federated learning. Moreover, we consider a variant of our algorithm that supports partial client participation, which makes the method more practical.
A Saliency based Feature Fusion Model for EEG Emotion Estimation

Among the different modalities to assess emotion, electroencephalogram (EEG), representing the electrical brain activity, achieved motivating results over the last decade. Emotion estimation from EEG could help in the diagnosis or rehabilitation of certain diseases. In this paper, we propose a dual model considering two different representations of EEG feature maps: 1) a sequential based representation of EEG band power, 2) an image-based representation of the feature vectors. We also propose an innovative method to combine the information based on a saliency analysis of the image-based model to promote joint learning of both model parts. The model has been evaluated on four publicly available datasets: SEED-IV, SEED, DEAP and MPED. The achieved results outperform results from state-of-the-art approaches for three of the proposed datasets with a lower standard deviation that reflects higher stability. For sake of reproducibility, the codes and models proposed in this paper are available at https://github.com/VDelv/Emotion-EEG.
Variational multiple shooting for Bayesian ODEs with Gaussian processes

Recent machine learning advances have proposed black-box estimation of unknown continuous-time system dynamics directly from data. However, earlier works are based on approximative ODE solutions or point estimates. We propose a novel Bayesian nonparametric model that uses Gaussian processes to infer posteriors of unknown ODE systems directly from data. We derive sparse variational inference with decoupled functional sampling to represent vector field posteriors. We also introduce a probabilistic shooting augmentation to enable efficient inference from arbitrarily long trajectories. The method demonstrates the benefit of computing vector field posteriors, with predictive uncertainty scores outperforming alternative methods on multiple ODE learning tasks.
MGA-VQA: Multi-Granularity Alignment for Visual Question Answering

Learning to answer visual questions is a challenging task since the multi-modal inputs are within two feature spaces. Moreover, reasoning in visual question answering requires the model to understand both image and question, and align them in the same space, rather than simply memorize statistics about the question-answer pairs. Thus, it is essential to find component connections between different modalities and within each modality to achieve better attention. Previous works learned attention weights directly on the features. However, the improvement is limited since these two modality features are in two domains: image features are highly diverse, lacking structure and grammatical rules as language, and natural language features have a higher probability of missing detailed information. To better learn the attention between visual and text, we focus on how to construct input stratification and embed structural information to improve the alignment between different level components. We propose Multi-Granularity Alignment architecture for Visual Question Answering task (MGA-VQA), which learns intra- and inter-modality correlations by multi-granularity alignment, and outputs the final result by the decision fusion module. In contrast to previous works, our model splits alignment into different levels to achieve learning better correlations without needing additional data and annotations. The experiments on the VQA-v2 and GQA datasets demonstrate that our model significantly outperforms non-pretrained state-of-the-art methods on both datasets without extra pretraining data and annotations. Moreover, it even achieves better results over the pre-trained methods on GQA.
Viscoelastodynamics of Swelling Porous Solids at Large Strains by an Eulerian Approach

A model of saturated hyperelastic porous solids at large strains is formulated and analysed. The material response is assumed to be of a viscoelastic Kelvin-Voigt type and inertial effects are considered, too. The flow of the diffusant is driven by the gradient of the chemical potential and is coupled to the mechanics via the occurrence of swelling and squeezing. Buoyancy effects due to the evolving mass density in a gravity field are covered. Higher-order viscosity is also included, allowing for physically relevant stored energies and local invertibility of the deformation. The whole system is formulated in a fully Eulerian form in terms of rates. The energetics of the model is discussed and the existence and regularity of weak solutions is proved by a combined regularization-Galerkin approximation argument.
Fast generation of spin squeezing via resonant spin-boson coupling

We propose protocols for the creation of useful entangled states in a system of spins collectively coupled to a bosonic mode, directly applicable to trapped- ion and cavity QED setups. The protocols use coherent manipulations of the resonant spin-boson interactions naturally arising in these systems to prepare spin squeezed states exponentially fast in time. The resonance condition harnesses the full spin-boson coupling and thus avoids the slower timescales when operating in the off-resonance regime. We demonstrate the robustness of the protocols by analyzing the effects of natural sources of decoherence in these systems and show their advantage compared to more standard slower approaches where entanglement is generated with off-resonant spin-boson interactions.
Correlations, Information Backflow, and Objectivity in a Class of Pure Dephasing Models

We critically examine the role that correlations established between a system and fragments of its environment play in characterising the ensuing dynamics. We employ a dephasing model with different initial conditions, where the state of the initial environment represents a tunable degree of freedom that qualitatively and quantitatively affects the correlation profiles, but nevertheless results in the same reduced dynamics for the system. We apply recently developed tools for the characterisation of non-Markovianity to carefully assess the role that correlations, as quantified by the (quantum) Jensen–Shannon divergence and relative entropy, as well as changes in the environmental state, play in whether the conditions for classical objectivity within the quantum Darwinism paradigm are met. We demonstrate that for precisely the same non-Markovian reduced dynamics of the system arising from different microscopic models, some exhibit quantum Darwinistic features, while others show that no meaningful notion of classical objectivity is present. Furthermore, our results highlight that the non-Markovian nature of an environment does not a priori prevent a system from redundantly proliferating relevant information, but rather it is the system’s ability to establish the requisite correlations that is the crucial factor in the manifestation of classical objectivity.
Bicategorical type theory: semantics and syntax

We develop semantics and syntax for bicategorical type theory. Bicategorical type theory features contexts, types, terms, and directed reductions between terms. This type theory is naturally interpreted in a class of structured bicategories. We start by developing the semantics, in the form of comprehension bicategories. Examples of comprehension bicategories are plentiful; we study both specific examples as well as classes of examples constructed from other data. From the notion of comprehension bicategory, we extract the syntax of bicategorical type theory, that is, judgment forms and structural inference rules. We prove soundness of the rules by giving an interpretation in any comprehension bicategory. The semantic aspects of our work are fully checked in the Coq proof assistant, based on the UniMath library.
Competition over data: how does data purchase affect users?

As machine learning (ML) is deployed by many competing service providers, the underlying ML predictors also compete against each other, and it is increasingly important to understand the impacts and biases from such competition. In this paper, we study what happens when the competing predictors can acquire additional labeled data to improve their prediction quality. We introduce a new environment that allows ML predictors to use active learning algorithms to purchase labeled data within their budgets while competing against each other to attract users. Our environment models a critical aspect of data acquisition in competing systems which has not been well-studied before. We found that the overall performance of an ML predictor improves when predictors can purchase additional labeled data. Surprisingly, however, the quality that users experience -- i.e. the accuracy of the predictor selected by each user -- can decrease even as the individual predictors get better. We show that this phenomenon naturally arises due to a trade-off whereby competition pushes each predictor to specialize in a subset of the population while data purchase has the effect of making predictors more uniform. We support our findings with both experiments and theories.
A Cooperation-Aware Lane Change Method for Autonomous Vehicles

Lane change for autonomous vehicles (AVs) is an important but challenging task in complex dynamic traffic environments. Due to difficulties in guarantee safety as well as a high efficiency, AVs are inclined to choose relatively conservative strategies for lane change. To avoid the conservatism, this paper presents a cooperation-aware lane change method utilizing interactions between vehicles. We first propose an interactive trajectory prediction method to explore possible cooperations between an AV and the others. Further, an evaluation is designed to make a decision on lane change, in which safety, efficiency and comfort are taken into consideration. Thereafter, we propose a motion planning algorithm based on model predictive control (MPC), which incorporates AV's decision and surrounding vehicles' interactive behaviors into constraints so as to avoid collisions during lane change. Quantitative testing results show that compared with the methods without an interactive prediction, our method enhances driving efficiencies of the AV and other vehicles by 14.8$\%$ and 2.6$\%$ respectively, which indicates that a proper utilization of vehicle interactions can effectively reduce the conservatism of the AV and promote the cooperation between the AV and others.
One Shot PACS: Patient Specific Anatomic Context and Shape Prior Aware Recurrent Registration-Segmentation of Longitudinal Thoracic Cone Beam CTs

Image-guided adaptive lung radiotherapy requires accurate tumor and organs segmentation from during treatment cone-beam CT (CBCT) images. Thoracic CBCTs are hard to segment because of low soft-tissue contrast, imaging artifacts, respiratory motion, and large treatment induced intra-thoracic anatomic changes. Hence, we developed a novel Patient-specific Anatomic Context and Shape prior or PACS-aware 3D recurrent registration-segmentation network for longitudinal thoracic CBCT segmentation. Segmentation and registration networks were concurrently trained in an end-to-end framework and implemented with convolutional long-short term memory models. The registration network was trained in an unsupervised manner using pairs of planning CT (pCT) and CBCT images and produced a progressively deformed sequence of images. The segmentation network was optimized in a one-shot setting by combining progressively deformed pCT (anatomic context) and pCT delineations (shape context) with CBCT images. Our method, one-shot PACS was significantly more accurate (p < 0.001) for tumor (DSC of 0.83 ± 0.08, surface DSC [sDSC] of 0.97 ± 0.06, and Hausdorff distance at 95th percentile [HD95] of 3.97±3.02mm) and the esophagus (DSC of 0.78 ± 0.13, sDSC of 0.90±0.14, HD95 of 3.22±2.02) segmentation than multiple methods. Ablation tests and comparative experiments were also done.
Skim: Skipping Memory Lstm for Low-Latency Real-Time Continuous Speech Separation

Continuous speech separation for meeting pre-processing has recently become a focused research topic. Compared to the data in utterance-level speech separation, the meeting-style audio stream lasts longer, has an uncertain number of speakers. We adopt the time-domain speech separation method and the recently proposed Graph-PIT to build a super low-latency online speech separation model, which is very important for the real application. The low-latency time-domain encoder with a small stride leads to an extremely long feature sequence. We proposed a simple yet efficient model named Skipping Memory (SkiM) for the long sequence modeling. Experimental results show that SkiM achieves on par or even better separation performance than DPRNN. Meanwhile, the computational cost of SkiM is reduced by 75% compared to DPRNN. The strong long sequence modeling capability and low computational cost make SkiM a suitable model for online CSS applications. Our fastest real-time model gets 17.1 dB signal-to-distortion (SDR) improvement with less than 1-millisecond latency in the simulated meeting-style evaluation.
$N\times N$ Matrix Time-/Band- Limiting examples

Time- and band-limiting in the context of orthogonal polynomials has been studied since the 1980's. It involves finding differential or difference operators with special commutative properties. More recently this topic has been generalized to the cases were the orthogonal polynomials are matrix-valued. Leading to differential or difference operators with matrix coefficients. So far the only explicit examples of such operators were $2\times 2$ matrices. In this paper we give a number of $N\times N$ examples and a counterexample to illustrate the role that strong Pearson equations can play in finding such examples.