OpenRaiser/Intern-Atlas huggingface dataset:
Conditional independence by typing
Venue: arXiv cs.PL 2022
Year: 2022

A central goal of probabilistic programming languages (PPLs) is to separate modelling from inference. However, this goal is hard to achieve in practice. Users are often forced to re-write their models in order to improve efficiency of inference or meet restrictions imposed by the PPL. Conditional independence (CI) relationships among parameters are a crucial aspect of probabilistic models that capture a qualitative summary of the specified model and can facilitate more efficient inference. We present an information flow type system for probabilistic programming that captures conditional independence (CI) relationships, and show that, for a well-typed program in our system, the distribution it implements is guaranteed to have certain CI-relationships. Further, by using type inference, we can statically deduce which CI-properties are present in a specified model. As a practical application, we consider the problem of how to perform inference on models with mixed discrete and continuous parameters. Inference on such models is challenging in many existing PPLs, but can be improved through a workaround, where the discrete parameters are used implicitly, at the expense of manual model re-writing. We present a source-to-source semantics-preserving transformation, which uses our CI-type system to automate this workaround by eliminating the discrete parameters from a probabilistic program. The resulting program can be seen as a hybrid inference algorithm on the original program, where continuous parameters can be drawn using efficient gradient-based inference methods, while the discrete parameters are inferred using variable elimination. We implement our CI-type system and its example application in SlicStan: a compositional variant of Stan.
Geometric Separability using Orthogonal Objects
Venue: arXiv cs.CG 2022
Year: 2022

Given a bichromatic point set $P=\textbf{R} \cup \textbf{B}$ of red and blue points, a separator is an object of a certain type that separates $\textbf{R}$ and $\textbf{B}$. We study the geometric separability problem when the separator is a) rectangular annulus of fixed orientation b) rectangular annulus of arbitrary orientation c) square annulus of fixed orientation d) orthogonal convex polygon. In this paper, we give polynomial time algorithms to construct separators of each of the above type that also optimizes a given parameter.
Performance Analysis of Distributed Intelligent Reflective Surfaces for Wireless Communications
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, a comprehensive performance analysis of a distributed intelligent reflective surfaces (IRSs)-aided communication system is presented. First, the optimal signal-to-noise ratio (SNR), which is attainable through the direct and reflected channels, is quantified by controlling the phase shifts of the distributed IRS. Next, this optimal SNR is statistically characterized by deriving tight approximations to the exact probability density function (PDF) and cumulative distribution function (CDF) for Nakagami-$m$ fading. The accuracy/tightness of this statistical characterization is investigated by deriving the Kullback-Leibler divergence. Our PDF/CDF analysis is used to derive tight approximations/bounds for the outage probability, achievable rate, and average symbol error rate (SER) in closed-form. To obtain useful insights, the asymptotic outage probability and average SER are derived for the high SNR regime. Thereby, the achievable diversity order and array gains are quantified. Our asymptotic performance analysis reveals that the diversity order can be boosted by using distributed passive IRSs without generating additional electromagnetic (EM) waves via active radio frequency chains. Our asymptotic rate analysis shows that the lower and upper rate bounds converge to an asymptotic limit in large reflective element regime. Our analysis is validated via Monte-Carlo simulations. We present a rigorous set of numerical results to investigate the performance gains of the proposed system model. Our analytical and numerical results reveal that the performance of single-input single-output wireless systems can be boosted by recycling the EM waves generated by a transmitter through distributed passive IRS reflections to enable constructive signal combining at a receiver.
Measuring Association Between Labels and Free-Text Rationales
Venue: arXiv cs.CL 2022
Year: 2022

In interpretable NLP, we require faithful rationales that reflect the model's decision-making process for an explained instance. While prior work focuses on extractive rationales (a subset of the input words), we investigate their less-studied counterpart: free-text natural language rationales. We demonstrate that pipelines, existing models for faithful extractive rationalization on information-extraction style tasks, do not extend as reliably to "reasoning" tasks requiring free-text rationales. We turn to models that jointly predict and rationalize, a class of widely used high-performance models for free-text rationalization whose faithfulness is not yet established. We define label-rationale association as a necessary property for faithfulness: the internal mechanisms of the model producing the label and the rationale must be meaningfully correlated. We propose two measurements to test this property: robustness equivalence and feature importance agreement. We find that state-of-the-art T5-based joint models exhibit both properties for rationalizing commonsense question-answering and natural language inference, indicating their potential for producing faithful free-text rationales.
CaM-Gen:Causally-aware Metric-guided Text Generation
Venue: arXiv cs.CL 2022
Year: 2022

Content is created for a well-defined purpose, often described by a metric or signal represented in the form of structured information. The relationship between the goal (metrics) of target content and the content itself is non-trivial. While large-scale language models show promising text generation capabilities, guiding the generated text with external metrics is challenging. These metrics and content tend to have inherent relationships and not all of them may be of consequence. We introduce CaM-Gen: Causally aware Generative Networks guided by user-defined target metrics incorporating the causal relationships between the metric and content features. We leverage causal inference techniques to identify causally significant aspects of a text that lead to the target metric and then explicitly guide generative models towards these by a feedback mechanism. We propose this mechanism for variational autoencoder and Transformer-based generative models. The proposed models beat baselines in terms of the target metric control while maintaining fluency and language quality of the generated text. To the best of our knowledge, this is one of the early attempts at controlled generation incorporating a metric guide using causal inference.
Deductive Stability Proofs for Ordinary Differential Equations
Venue: arXiv cs.LO 2022
Year: 2022

Stability is required for real world controlled systems as it ensures that those systems can tolerate small, real world perturbations around their desired operating states. This paper shows how stability for continuous systems modeled by ordinary differential equations (ODEs) can be formally verified in differential dynamic logic (dL). The key insight is to specify ODE stability by suitably nesting the dynamic modalities of dL with first-order logic quantifiers. Elucidating the logical structure of stability properties in this way has three key benefits: i) it provides a flexible means of formally specifying various stability properties of interest, ii) it yields rigorous proofs of those stability properties from dL's axioms with dL's ODE safety and liveness proof principles, and iii) it enables formal analysis of the relationships between various stability properties which, in turn, inform proofs of those properties. These benefits are put into practice through an implementation of stability proofs for several examples in KeYmaera X, a hybrid systems theorem prover based on dL.
Correlation Robust Influence Maximization
Venue: arXiv cs.SI 2022
Year: 2022

We propose a distributionally robust model for the influence maximization problem. Unlike the classic independent cascade model \citep{kempe2003maximizing}, this model's diffusion process is adversarially adapted to the choice of seed set. Hence, instead of optimizing under the assumption that all influence relationships in the network are independent, we seek a seed set whose expected influence under the worst correlation, i.e. the "worst-case, expected influence", is maximized. We show that this worst-case influence can be efficiently computed, and though the optimization is NP-hard, a ($1 - 1/e$) approximation guarantee holds. We also analyze the structure to the adversary's choice of diffusion process, and contrast with established models. Beyond the key computational advantages, we also highlight the extent to which the independence assumption may cost optimality, and provide insights from numerical experiments comparing the adversarial and independent cascade model.
Differentiable Channel Sparsity Search via Weight Sharing within Filters
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we propose the differentiable channel sparsity search (DCSS) for convolutional neural networks. Unlike traditional channel pruning algorithms which require users to manually set prune ratios for each convolutional layer, DCSS automatically searches the optimal combination of sparsities. Inspired by the differentiable architecture search (DARTS), we draw lessons from the continuous relaxation and leverage the gradient information to balance the computational cost and metrics. Since directly applying the scheme of DARTS causes shape mismatching and excessive memory consumption, we introduce a novel technique called weight sharing within filters. This technique elegantly eliminates the problem of shape mismatching with negligible additional resources. We conduct comprehensive experiments on not only image classification but also find-grained tasks including semantic segmentation and image super resolution to verify the effectiveness of DCSS. Compared with previous network pruning approaches, DCSS achieves state-of-the-art results for image classification. Experimental results of semantic segmentation and image super resolution indicate that task-specific search achieves better performance than transferring slim models, demonstrating the wide applicability and high efficiency of DCSS.
Language for Description of Worlds
Venue: arXiv cs.AI 2022
Year: 2022

We will reduce the task of creating AI to the task of finding an appropriate language for description of the world. This will not be a programing language because programing languages describe only computable functions, while our language will describe a somewhat broader class of functions. Another specificity of this language will be that the description will consist of separate modules. This will enable us look for the description of the world automatically such that we discover it module after module. Our approach to the creation of this new language will be to start with a particular world and write the description of that particular world. The point is that the language which can describe this particular world will be appropriate for describing any world.
Representation Decomposition for Image Manipulation and Beyond
Venue: arXiv cs.CV 2022
Year: 2022

Representation disentanglement aims at learning interpretable features, so that the output can be recovered or manipulated accordingly. While existing works like infoGAN and AC-GAN exist, they choose to derive disjoint attribute code for feature disentanglement, which is not applicable for existing/trained generative models. In this paper, we propose a decomposition-GAN (dec-GAN), which is able to achieve the decomposition of an existing latent representation into content and attribute features. Guided by the classifier pre-trained on the attributes of interest, our dec-GAN decomposes the attributes of interest from the latent representation, while data recovery and feature consistency objectives enforce the learning of our proposed method. Our experiments on multiple image datasets confirm the effectiveness and robustness of our dec-GAN over recent representation disentanglement models.
Unified greedy approximability beyond submodular maximization
Venue: arXiv cs.DM 2022
Year: 2022

We consider classes of objective functions of cardinality constrained maximization problems for which the greedy algorithm guarantees a constant approximation. We propose the new class of $γ$-$α$-augmentable functions and prove that it encompasses several important subclasses, such as functions of bounded submodularity ratio, $α$-augmentable functions, and weighted rank functions of an independence system of bounded rank quotient - as well as additional objective functions for which the greedy algorithm yields an approximation. For this general class of functions, we show a tight bound of $\fracαγ\cdot\frac{\mathrm{e}^α}{\mathrm{e}^α-1}$ on the approximation ratio of the greedy algorithm that tightly interpolates between bounds from the literature for functions of bounded submodularity ratio and for $α$-augmentable functions. In paritcular, as a by-product, we close a gap left in [Math.Prog., 2020] by obtaining a tight lower bound for $α$-augmentable functions for all $α\geq1$. For weighted rank functions of independence systems, our tight bound becomes $\fracαγ$, which recovers the known bound of $1/q$ for independence systems of rank quotient at least $q$.
Paralinguistic Privacy Protection at the Edge
Venue: arXiv cs.CL 2022
Year: 2022

Voice user interfaces and digital assistants are rapidly entering our lives and becoming singular touch points spanning our devices. These always-on services capture and transmit our audio data to powerful cloud services for further processing and subsequent actions. Our voices and raw audio signals collected through these devices contain a host of sensitive paralinguistic information that is transmitted to service providers regardless of deliberate or false triggers. As our emotional patterns and sensitive attributes like our identity, gender, well-being, are easily inferred using deep acoustic models, we encounter a new generation of privacy risks by using these services. One approach to mitigate the risk of paralinguistic-based privacy breaches is to exploit a combination of cloud-based processing with privacy-preserving, on-device paralinguistic information learning and filtering before transmitting voice data. In this paper we introduce EDGY, a configurable, lightweight, disentangled representation learning framework that transforms and filters high-dimensional voice data to identify and contain sensitive attributes at the edge prior to offloading to the cloud. We evaluate EDGY's on-device performance and explore optimization techniques, including model quantization and knowledge distillation, to enable private, accurate and efficient representation learning on resource-constrained devices. Our results show that EDGY runs in tens of milliseconds with 0.2% relative improvement in "zero-shot" ABX score or minimal performance penalties of approximately 5.95% word error rate (WER) in learning linguistic representations from raw voice signals, using a CPU and a single-core ARM processor without specialized hardware.
RealAnt: An Open-Source Low-Cost Quadruped for Education and Research in Real-World Reinforcement Learning
Venue: arXiv cs.RO 2022
Year: 2022

Current robot platforms available for research are either very expensive or unable to handle the abuse of exploratory controls in reinforcement learning. We develop RealAnt, a minimal low-cost physical version of the popular `Ant' benchmark used in reinforcement learning. RealAnt costs only $\sim$350 EUR (\$410) in materials and can be assembled in less than an hour. We validate the platform with reinforcement learning experiments and provide baseline results on a set of benchmark tasks. We demonstrate that the RealAnt robot can learn to walk from scratch from less than 10 minutes of experience. We also provide simulator versions of the robot (with the same dimensions, state-action spaces, and delayed noisy observations) in the MuJoCo and PyBullet simulators. We open-source hardware designs, supporting software, and baseline results for educational use and reproducible research.
B-GAP: Behavior-Rich Simulation and Navigation for Autonomous Driving
Venue: arXiv cs.RO 2022
Year: 2022

We address the problem of ego-vehicle navigation in dense simulated traffic environments populated by road agents with varying driver behaviors. Navigation in such environments is challenging due to unpredictability in agents' actions caused by their heterogeneous behaviors. We present a new simulation technique consisting of enriching existing traffic simulators with behavior-rich trajectories corresponding to varying levels of aggressiveness. We generate these trajectories with the help of a driver behavior modeling algorithm. We then use the enriched simulator to train a deep reinforcement learning (DRL) policy that consists of a set of high-level vehicle control commands and use this policy at test time to perform local navigation in dense traffic. Our policy implicitly models the interactions between traffic agents and computes safe trajectories for the ego-vehicle accounting for aggressive driver maneuvers such as overtaking, over-speeding, weaving, and sudden lane changes. Our enhanced behavior-rich simulator can be used for generating datasets that consist of trajectories corresponding to diverse driver behaviors and traffic densities, and our behavior-based navigation scheme can be combined with state-of-the-art navigation algorithms.
Performative Prediction in a Stateful World
Venue: arXiv cs.LG 2022
Year: 2022

Deployed supervised machine learning models make predictions that interact with and influence the world. This phenomenon is called performative prediction by Perdomo et al. (ICML 2020). It is an ongoing challenge to understand the influence of such predictions as well as design tools so as to control that influence. We propose a theoretical framework where the response of a target population to the deployed classifier is modeled as a function of the classifier and the current state (distribution) of the population. We show necessary and sufficient conditions for convergence to an equilibrium of two retraining algorithms, repeated risk minimization and a lazier variant. Furthermore, convergence is near an optimal classifier. We thus generalize results of Perdomo et al., whose performativity framework does not assume any dependence on the state of the target population. A particular phenomenon captured by our model is that of distinct groups that acquire information and resources at different rates to be able to respond to the latest deployed classifier. We study this phenomenon theoretically and empirically.
Quantum-Inspired Algorithms from Randomized Numerical Linear Algebra
Venue: arXiv cs.DS 2022
Year: 2022

We create classical (non-quantum) dynamic data structures supporting queries for recommender systems and least-squares regression that are comparable to their quantum analogues. De-quantizing such algorithms has received a flurry of attention in recent years; we obtain sharper bounds for these problems. More significantly, we achieve these improvements by arguing that the previous quantum-inspired algorithms for these problems are doing leverage or ridge-leverage score sampling in disguise; these are powerful and standard techniques in randomized numerical linear algebra. With this recognition, we are able to employ the large body of work in numerical linear algebra to obtain algorithms for these problems that are simpler or faster (or both) than existing approaches. Our experiments demonstrate that the proposed data structures also work well on real-world datasets.
PRVNet: A Novel Partially-Regularized Variational Autoencoders for Massive MIMO CSI Feedback
Venue: arXiv cs.NI 2022
Year: 2022

In a multiple-input multiple-output frequency-division duplexing (MIMO-FDD) system, the user equipment (UE) sends the downlink channel state information (CSI) to the base station to report link status. Due to the complexity of MIMO systems, the overhead incurred in sending this information negatively affects the system bandwidth. Although this problem has been widely considered in the literature, prior work generally assumes an ideal feedback channel. In this paper, we introduce PRVNet, a neural network architecture inspired by variational autoencoders (VAE) to compress the CSI matrix before sending it back to the base station under noisy channel conditions. Moreover, we propose a customized loss function that best suits the special characteristics of the problem being addressed. We also introduce an additional regularization hyperparameter for the learning objective, which is crucial for achieving competitive performance. In addition, we provide an efficient way to tune this hyperparameter using KL-annealing. Experimental results show the proposed model outperforms the benchmark models including two deep learning-based models in a noise-free feedback channel assumption. In addition, the proposed model achieves an outstanding performance under different noise levels for additive white Gaussian noise feedback channels.
Detecting Outliers with Foreign Patch Interpolation
Venue: arXiv cs.CV 2022
Year: 2022

In medical imaging, outliers can contain hypo/hyper-intensities, minor deformations, or completely altered anatomy. To detect these irregularities it is helpful to learn the features present in both normal and abnormal images. However this is difficult because of the wide range of possible abnormalities and also the number of ways that normal anatomy can vary naturally. As such, we leverage the natural variations in normal anatomy to create a range of synthetic abnormalities. Specifically, the same patch region is extracted from two independent samples and replaced with an interpolation between both patches. The interpolation factor, patch size, and patch location are randomly sampled from uniform distributions. A wide residual encoder decoder is trained to give a pixel-wise prediction of the patch and its interpolation factor. This encourages the network to learn what features to expect normally and to identify where foreign patterns have been introduced. The estimate of the interpolation factor lends itself nicely to the derivation of an outlier score. Meanwhile the pixel-wise output allows for pixel- and subject- level predictions using the same model.
A Review of the Family of Artificial Fish Swarm Algorithms: Recent Advances and Applications
Venue: arXiv cs.NE 2022
Year: 2022

The Artificial Fish Swarm Algorithm (AFSA) is inspired by the ecological behaviors of fish schooling in nature, viz., the preying, swarming and following behaviors. Owing to a number of salient properties, which include flexibility, fast convergence, and insensitivity to the initial parameter settings, the family of AFSA has emerged as an effective Swarm Intelligence (SI) methodology that has been widely applied to solve real-world optimization problems. Since its introduction in 2002, many improved and hybrid AFSA models have been developed to tackle continuous, binary, and combinatorial optimization problems. This paper aims to present a concise review of the continuous AFSA, encompassing the original ASFA, its improvements and hybrid models, as well as their associated applications. We focus on articles published in high-quality journals since 2013. Our review provides insights into AFSA parameters modifications, procedures and sub-functions. The main reasons for these enhancements and the comparison results with other hybrid methods are discussed. In addition, hybrid, multi-objective and dynamic AFSA models that have been proposed to solve continuous optimization problems are elucidated. We also analyse possible AFSA enhancements and highlight future research directions for advancing AFSA-based models.