Miracle-dz/UniFAR-FTUs huggingface dataset:
Differentiable Factor Graph Optimization for Learning Smoothers

A recent line of work has shown that end-to-end optimization of Bayesian filters can be used to learn state estimators for systems whose underlying models are difficult to hand-design or tune, while retaining the core advantages of probabilistic state estimation. As an alternative approach for state estimation in these settings, we present an end-to-end approach for learning state estimators modeled as factor graph-based smoothers. By unrolling the optimizer we use for maximum a posteriori inference in these probabilistic graphical models, our method is able to learn probabilistic system models in the full context of an overall state estimator, while also taking advantage of the distinct accuracy and runtime advantages that smoothers offer over recursive filters. We study our approach using two fundamental state estimation problems, object tracking and visual odometry, where we demonstrate a significant improvement over existing baselines. Our work comes with an extensive code release, which includes training and evaluation scripts, as well as Python libraries for Lie theory and factor graph optimization: https://sites.google. com/view/diffsmoothing/.
DCAP: Deep Cross Attentional Product Network for User Response Prediction

User response prediction, which aims to predict the probability that a user will provide a predefined positive response in a given context such as clicking on an ad or purchasing an item, is crucial to many industrial applications such as online advertising, recommender systems, and search ranking. For these tasks and many other machine learning tasks, an indispensable part of success is feature engineering, where cross features are a significant type of feature transformations. However, due to the high dimensionality and super sparsity of the data collected in these tasks, handcrafting cross features is inevitably time expensive. Prior studies in predicting user response leveraged the feature interactions by enhancing feature vectors with products of features to model second-order or high-order cross features, either explicitly or implicitly. However, these existing methods can be hindered by not learning sufficient cross features due to model architecture limitations or modeling all high-order feature interactions with equal weights. Different features should contribute differently to the prediction, and not all cross features are with the same prediction power. This work aims to fill this gap by proposing a novel architecture Deep Cross Attentional Product Network (DCAP), which keeps cross network's benefits in modeling high-order feature interactions explicitly at the vector-wise level. By computing the inner product or outer product between attentional feature embeddings and original input embeddings as each layer's output, we can model cross features with a higher degree of order as the network's depth increases. We concatenate all the outputs from each layer, which further helps the model capture much information on cross features of different orders. Beyond that, it can differentiate the importance of different cross features in each network layer inspired by the multi-head attention mechanism and Product Neural Network (PNN), allowing practitioners to perform a more in-depth analysis of user behaviors. Additionally, our proposed model can be easily implemented and train in parallel. We conduct comprehensive experiments on three real-world datasets. The results have robustly demonstrated that our proposed model DCAP achieves superior prediction performance compared with the state-of-the-art models. Public codes are available at https://github.com/zachstarkk/DCAP.
Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

In real-world multi-agent systems, agents with different capabilities may join or leave without altering the team's overarching goals. Coordinating teams with such dynamic composition is challenging: the optimal team strategy varies with the composition. We propose COPA, a coach-player framework to tackle this problem. We assume the coach has a global view of the environment and coordinates the players, who only have partial views, by distributing individual strategies. Specifically, we 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players. We validate our methods on a resource collection task, a rescue game, and the StarCraft micromanagement tasks. We demonstrate zero-shot generalization to new team compositions. Our method achieves comparable or better performance than the setting where all players have a full view of the environment. Moreover, we see that the performance remains high even when the coach communicates as little as 13% of the time using the adaptive communication strategy.
Robust Doppler Filter Design for Joint RadCom Systems Based on Spatial Modulation

This paper analyses the effect of beampattern variation on clutter modulation for joint radar and communication (RadCom) systems based on spatial modulation. Besides, this paper considers the design of robust doppler filter for joint RadCom system when the receive temporal steering vector is mismatched with the desired temporal steering vector. This paper aims to maximize the worst Signal-to-Interference-plus-NoiseRatio (SINR) of the doppler filter in the doppler interval under the similarity constraint and constant white noise gain constraint. In order to solve the nonconvex optimization problem, we relax the original optimization problem to a semidefinite program (SDP), which is convex and can be solved using CVX toolbox. Lots of numerical results are presented to show the robustness of the designed doppler filter.
A Review of Intrusion Detection Systems and Their Evaluation in the IoT

Intrusion Detection Systems (IDS) are key components for securing critical infrastructures, capable of detecting malicious activities on networks or hosts. The procedure of implementing a IDS for Internet of Things (IoT) networks is not without challenges due to the variability of these systems and specifically the difficulty in accessing data. The specifics of these very constrained devices render the design of an IDS capable of dealing with the varied attacks a very challenging problem and a very active research subject. In the current state of literature, a number of approaches have been proposed to improve the efficiency of intrusion detection, catering to some of these limitations, such as resource constraints and mobility. In this article, we review works on IDS specifically for these kinds of devices from 2008 to 2018, collecting a total of 51 different IDS papers. We summarise the current themes of the field, summarise the techniques employed to train and deploy the IDSs and provide a qualitative evaluations of these approaches. While these works provide valuable insights and solutions for sub-parts of these constraints, we discuss the limitations of these solutions as a whole, in particular what kinds of attacks these approaches struggle to detect and the setup limitations that are unique to this kind of system. We find that although several paper claim novelty of their approach little inter paper comparisons have been made, that there is a dire need for sharing of datasets and almost no shared code repositories, consequently raising the need for a thorough comparative evaluation.
Frequency measurements beyond the Heisenberg time-energy limit with a single atom

The Heisenberg time-energy relation prevents determination of an atomic transition to better than the inverse of the measurement time. The relation generally applies to frequency estimation of a near-resonant field [1-3], since information on the field frequency can be used to infer the atomic transition [4, 5]. Here we demonstrate a frequency estimation technique that provides an uncertainty orders of magnitude below the Heisenberg limit with a single atom. With access to $N$ atoms, we propose a fundamental uncertainty limit improving as $\sqrt{N}$, regardless of whether entanglement is employed. We describe implementation of the quantum fourier transform to estimate an unknown frequency without using entanglement. A comparison to classical algorithms severely limits the benefit that quantum algorithms provide for frequency estimation and that entanglement provides to quantum sensing in general.
Discovery and Contextual Data Cleaning with Ontology Functional Dependencies

Functional Dependencies (FDs) define attribute relationships based on syntactic equality, and, when used in data cleaning, they erroneously label syntactically different but semantically equivalent values as errors. We explore dependency-based data cleaning with Ontology Functional Dependencies (OFDs), which express semantic attribute relationships such as synonyms and is-a hierarchies defined by an ontology. We study the theoretical foundations for OFDs, including sound and complete axioms and a linear-time inference procedure. We then propose an algorithm for discovering OFDs (exact ones and ones that hold with some exceptions) from data that uses the axioms to prune the search space. Towards enabling OFDs as data quality rules in practice, we study the problem of finding minimal repairs to a relation and ontology with respect to a set of OFDs. We demonstrate the effectiveness of our techniques on real datasets, and show that OFDs can significantly reduce the number of false positive errors in data cleaning techniques that rely on traditional FDs.
A time-domain nearfield frequency-invariant beamforming method

Most existing beamforming methods are frequency-domain methods, and are designed for enhancing a farfield target source over a narrow frequency band. They have found diverse applications and are still under active development. However, they struggle to achieve desired performance if the target source is in the nearfield with a broadband output. This paper proposes a time-domain nearfield frequency-invariant beamforming method. The time-domain implementation makes the beamformer output suitable for further use by real-time applications, the nearfield focusing enables the beamforming method to suppress an interference even if it is in the same direction as the target source, and the frequency-invariant beampattern makes the beamforming method suitable for enhancing the target source over a broad frequency band. These three features together make the beamforming method suitable for real-time broadband nearfield source enhancement, such as speech enhancement in room environments. The beamformer design process is separated from the sound field measurement process, and such that a designed beamformer applies to sensor arrays with various structures. The beamformer design process is further simplified by decomposing it into several independent parts. Simulation This work is sponsored by the Australian Research Council (ARC) Discovery Projects funding schemes with project numbers DP200100693. Fei Ma, Thushara D. Abhayapala, and Prasanga N. Samarasinghe are all with the Research School of Engineering, College of Engineering and Computer Science, The Australian National University, Canberra, ACT 2601, Australia (e-mail: fei.ma, Thushara.Abhayapala, prasanga.samarasinghe@anu.edu.au). Manuscript received xxx xxx, xxx; revised xxx xx, xxx. results confirm the performance of the proposed beamforming method.
“Worst-case” Microlensing in the Identification and Modeling of Lensed Quasars

Although microlensing of macrolensed quasars and supernovae provides unique opportunities for several kinds of investigations, it can add unwanted and sometimes substantial noise. While microlensing flux anomalies may be safely ignored for some observations, they severely limit others. “Worst-case” estimates can inform the decision whether or not to undertake an extensive examination of microlensing scenarios. Here, we report “worst-case” microlensing uncertainties for point sources lensed by singular isothermal potentials, parameterized by a convergence equal to the shear and by the stellar fraction. The results can be straightforwardly applied to nonisothermal potentials by utilizing the mass sheet degeneracy. We use microlensing maps to compute the fluctuations in image micromagnifications and estimate the stellar fraction at which the fluctuations are greatest for a given convergence. We find that the worst-case fluctuations happen at the stellar fraction κ⋆=1∣μmacro∣ . For macrominima, the fluctuations in both magnification and demagnification appear to be bounded (1.5 > Δm > −1.3, where Δm is the magnitude relative to the average macromagnification). Magnifications for macrosaddles are bounded as well (Δm > −1.7). In contrast, demagnifications for macrosaddles appear to have unbounded fluctuations as 1/μ macro → 0 and κ ⋆ → 0.
Convergence Rates of Gradient Methods for Convex Optimization in the Space of Measures

We study the convergence rate of Bregman gradient methods for convex optimization in the space of measures on a $d$-dimensional manifold. Under basic regularity assumptions, we show that the suboptimality gap at iteration $k$ is in $O(log(k)k^{--1})$ for multiplicative updates, while it is in $O(k^{--q/(d+q)})$ for additive updates for some $q \in {1, 2, 4}$ determined by the structure of the objective function. Our flexible proof strategy, based on approximation arguments, allows us to painlessly cover all Bregman Proximal Gradient Methods (PGM) and their acceleration (APGM) under various geometries such as the hyperbolic entropy and $L^p$ divergences. We also prove the tightness of our analysis with matching lower bounds and confirm the theoretical results with numerical experiments on low dimensional problems. Note that all these optimization methods must additionally pay the computational cost of discretization, which can be exponential in $d$.
Sparsity Prior Regularized Q-learning for Sparse Action Tasks

In many decision-making tasks, some speciﬁc actions are limited in their frequency or total amounts, such as “ﬁre” in the gunﬁght game and “buy/sell” in the stock trading. We name such actions as “sparse action”. Sparse action often plays a crucial role in achieving good performance. However, their Q-values, estimated by classical Bell-man update , usually suffer from a large estimation error due to the sparsity of their samples. The greedy policy could be greatly misled by the biased Q-function and takes sparse action aggressively, which leads to a huge sub-optimality. This paper constructs a reference distribution that assigns a low probability to sparse action and proposes a regularized objective with an explicit constraint to the reference distribution. Furthermore, we derive a regularized Bellman operator and a regularized optimal policy that can slow down the propagation of error and guide the agent to take sparse action more carefully. The experiment results demonstrate that our method achieves state-of-the-art performance on typical sparse action tasks.
Efficient calculations of the Mode-Resolved ab-initio thermal Conductivity in nanostructures

First-principles calculations of thermal transport in homogeneous materials have reached remarkable predicting power. Modeling deterministically phonon transport in nanostructures, however, poses novel challenges; notably, it entails solving as many algebraic equations as the number of combinations of wave vectors in the discretized Brillouen and polarizations. We show that, within the relaxation time approximation of the Boltzmann transport equation (BTE), this issue is resolved by interpolating the phonon distributions in the vectorial phonon mean free paths (MFP) space. The coupling between structure and mode-resolved heat transport is investigated in terms of angular-resolved bulk thermal conductivity and phonon suppression function, the latter being associated primarily to the material’s geometry. Our method, termed the anisotropic MFP-BTE (aMFP-BTE), allows for fast and accurate thermal conductivity calculations in nanomaterials regardless of the number of phonon branches and wave vectors. Furthermore, it naturally blends with first-principles thermal transport calculations, therefore allowing for multiscale, parameter-free simulations. We apply the aMFP-BTE to compute the mode-resolved effective thermal conductivity of porous Si membranes, achieving up to 50x speed with respect to the case with no interpolation. The proposed approach unlocks the engineering of novel nanostructures, with applications to thermoelectrics and heat management.
A sparse ADMM-based solver for linear MPC subject to terminal quadratic constraint

This paper presents a sparse solver based on the alternating direction method of multipliers algorithm for a linear model predictive control (MPC) formulation in which the terminal state is constrained to a given ellipsoid. The motivation behind this solver is to substitute the typical polyhedral invariant set used as a terminal constraint in many nominal and robust linear MPC formulations with an invariant set in the form of an ellipsoid, which is (typically) much easier to compute and results in an optimization problem with significantly fewer constraints, even for average-sized systems. However, this optimization problem is no longer the quadratic programming problem found in most linear MPC approaches, thus meriting the development of a tailored solver. The proposed solver is suitable for its use in embedded systems, since it is sparse, has a small memory footprint and requires no external libraries. We show the results of its implementation in an embedded system to control a simulated multivariable plant, comparing it against other alternatives.
Ring of attraction: overlapping directions of the dipole modulation of the CMB, the parity asymmetry, and kinematic dipole percolation zone

The largest anisotropy in the cosmic microwave background (CMB) is the 3 mK kinematic dipole reflecting our motion with respect to the CMB frame and pointed in the direction (l, b) = (264◦,+48◦) in Galactic coordinates. We introduce the concept of the ring of attraction (RA), which is orthogonal to the axis of the kinematic dipole. These directions overlap with the zone of percolation for the kinematic dipole, where its amplitude almost vanishes. We show that along this ring are oriented the directions of the dipole modulation of the CMB, and positions of the peaks responsible for generation of parity asymmetry. This coincidence is peculiar at greater than the 3 sigma level. We analyzed the “interaction” of low multipoles of the CMB with RA and showed that for odd modes there is a sequence of peaks in the RA direction. These peaks correlate with each other for different multipoles and result in mutual amplification of the odd ` signal for the first 30 multipoles. Our method sheds new light on the nature of parity asymmetry. It consists of the deficit of symmetrically located and equal in amplitude peaks in the CMB map in comparison with asymmetric peaks.
Learning to Route via Theory-Guided Residual Network

The heavy traffic and related issues have always been concerns for modern cities. With the help of deep learning and reinforcement learning, people have proposed various policies to solve these traffic-related problems, such as smart traffic signal control systems and taxi dispatching systems. People usually validate these policies in a city simulator, since directly applying them in the real city introduces real cost. However, these policies validated in the city simulator may fail in the real city if the simulator is significantly different from the real world. To tackle this problem, we need to build a real-like traffic simulation system. Therefore, in this paper, we propose to learn the human routing model, which is one of the most essential part in the traffic simulator. This problem has two major challenges. First, human routing decisions are determined by multiple factors, besides the common time and distance factor. Second, current historical routes data usually covers just a small portion of vehicles, due to privacy and device availability issues. To address these problems, we propose a theory-guided residual network model, where the theoretical part can emphasize the general principles for human routing decisions (e.g., fastest route), and the residual part can capture drivable condition preferences (e.g., local road or highway). Since the theoretical part is composed of traditional shortest path algorithms that do not need data to train, our residual network can learn human routing models from limited data. We have conducted extensive experiments on multiple real-world datasets to show the superior performance of our model, especially with small data. Besides, we have also illustrated why our model is better at recovering real routes through case studies.
Relation Classification with Entity Type Restriction

Relation classification aims to predict a relation between two entities in a sentence. The existing methods regard all relations as the candidate relations for the two entities in a sentence. These methods neglect the restrictions on candidate relations by entity types, which leads to some inappropriate relations being candidate relations. In this paper, we propose a novel paradigm, RElation Classification with ENtity Type restriction (RECENT), which exploits entity types to restrict candidate relations. Specially, the mutual restrictions of relations and entity types are formalized and introduced into relation classification. Besides, the proposed paradigm, RECENT, is model-agnostic. Based on two representative models GCN and SpanBERT respectively, RECENT_GCN and RECENT_SpanBERT are trained in RECENT. Experimental results on a standard dataset indicate that RECENT improves the performance of GCN and SpanBERT by 6.9 and 4.4 F1 points, respectively. Especially, RECENT_SpanBERT achieves a new state-of-the-art on TACRED.
An analogue of quasi-transitivity for edge-coloured graphs

We extend the notion of quasi-transitive orientations of graphs to 2-edge-coloured graphs. By relating quasi-transitive $2$-edge-colourings to an equivalence relation on the edge set of a graph, we classify those graphs that admit a quasi-transitive $2$-edge-colouring. As a contrast to Ghouil\'{a}-Houri's classification of quasi-transitively orientable graphs as comparability graphs, we find quasi-transitively $2$-edge-colourable graphs do not admit a forbiddden subgraph characterization. Restricting the problem to comparability graphs, we show that the family of uniquely quasi-transitively orientable comparability graphs is exactly the family of comparabilty graphs that admit no quasi-transitive $2$-edge-colouring.
Social Behavior and Mental Health: A Snapshot Survey under COVID-19 Pandemic

Online social media provides a channel for monitoring people's social behaviors and their mental distress. Due to the restrictions imposed by COVID-19 people are increasingly using online social networks to express their feelings. Consequently, there is a significant amount of diverse user-generated social media content. However, COVID-19 pandemic has changed the way we live, study, socialize and recreate and this has affected our well-being and mental health problems. There are growing researches that leverage online social media analysis to detect and assess user's mental status. In this paper, we survey the literature of social media analysis for mental disorders detection, with a special focus on the studies conducted in the context of COVID-19 during 2020-2021. Firstly, we classify the surveyed studies in terms of feature extraction types, varying from language usage patterns to aesthetic preferences and online behaviors. Secondly, we explore detection methods used for mental disorders detection including machine learning and deep learning detection methods. Finally, we discuss the challenges of mental disorder detection using social media data, including the privacy and ethical concerns, as well as the technical challenges of scaling and deploying such systems at large scales, and discuss the learnt lessons over the last few years.