OpenRaiser/Intern-Atlas huggingface dataset:
Recurrent networks, hidden states and beliefs in partially observable environments
Venue: arXiv cs.LG 2022
Year: 2022

Reinforcement learning aims to learn optimal policies from interaction with environments whose dynamics are unknown. Many methods rely on the approximation of a value function to derive near-optimal policies. In partially observable environments, these functions depend on the complete sequence of observations and past actions, called the history. In this work, we show empirically that recurrent neural networks trained to approximate such value functions internally filter the posterior probability distribution of the current state given the history, called the belief. More precisely, we show that, as a recurrent neural network learns the Q-function, its hidden states become more and more correlated with the beliefs of state variables that are relevant to optimal control. This correlation is measured through their mutual information. In addition, we show that the expected return of an agent increases with the ability of its recurrent architecture to reach a high mutual information between its hidden states and the beliefs. Finally, we show that the mutual information between the hidden states and the beliefs of variables that are irrelevant for optimal control decreases through the learning process. In summary, this work shows that in its hidden states, a recurrent neural network approximating the Q-function of a partially observable environment reproduces a sufficient statistic from the history that is correlated to the relevant part of the belief for taking optimal actions.
Strong authentication on smart wireless devices
Venue: arXiv cs.CR 2022
Year: 2022

The rapid deployment of wireless technologies has given rise to the current situation where mobile phones and other wireless devices have become essential elements in all types of activities, including in the home. In particular, smartphones and laptops are used for wirelessly sharing photos and documents, playing games, browsing websites, and viewing multimedia, for example. This work describes a proposal for both desktop and mobile applications that use Identity-Based Cryptography (IBC) to protect communications between smart wireless devices in the home. It combines the use of IBC for Wi-Fi and Bluetooth communication, with the promising Near Field Communication (NFC) technology for secure authentication. The proposed scheme involves NFC pairing to establish as public key a piece of information linked to the device, such as a phone number or an IP address. In this way, such information can be then used in an IBC scheme for peer-to-peer communication. This is a work in progress, but preliminary implementations of prototypes on several mobile platforms have already produced promising results.
Triple Sparsification of Graph Convolutional Networks without Sacrificing the Accuracy
Venue: arXiv cs.LG 2022
Year: 2022

Graph Neural Networks (GNNs) are widely used to perform different machine learning tasks on graphs. As the size of the graphs grows, and the GNNs get deeper, training and inference time become costly in addition to the memory requirement. Thus, without sacrificing accuracy, graph sparsification, or model compression becomes a viable approach for graph learning tasks. A few existing techniques only study the sparsification of graphs and GNN models. In this paper, we develop a SparseGCN pipeline to study all possible sparsification in GNN. We provide a theoretical analysis and empirically show that it can add up to 11.6\% additional sparsity to the embedding matrix without sacrificing the accuracy of the commonly used benchmark graph datasets.
HSIC-InfoGAN: Learning Unsupervised Disentangled Representations by Maximising Approximated Mutual Information
Venue: arXiv cs.CV 2022
Year: 2022

Learning disentangled representations requires either supervision or the introduction of specific model designs and learning constraints as biases. InfoGAN is a popular disentanglement framework that learns unsupervised disentangled representations by maximising the mutual information between latent representations and their corresponding generated images. Maximisation of mutual information is achieved by introducing an auxiliary network and training with a latent regression loss. In this short exploratory paper, we study the use of the Hilbert-Schmidt Independence Criterion (HSIC) to approximate mutual information between latent representation and image, termed HSIC-InfoGAN. Directly optimising the HSIC loss avoids the need for an additional auxiliary network. We qualitatively compare the level of disentanglement in each model, suggest a strategy to tune the hyperparameters of HSIC-InfoGAN, and discuss the potential of HSIC-InfoGAN for medical applications.
An FPGA framework for Interferometric Vision-Based Navigation (iVisNav)
Venue: arXiv cs.AR 2022
Year: 2022

Interferometric Vision-Based Navigation (iVisNav) is a novel optoelectronic sensor for autonomous proximity operations. iVisNav employs laser emitting structured beacons and precisely characterizes six degrees of freedom relative motion rates by measuring changes in the phase of the transmitted laser pulses. iVisNav's embedded package must efficiently process high frequency dynamics for robust sensing and estimation. A new embedded system for least squares-based rate estimation is developed in this paper. The resulting system is capable of interfacing with the photonics and implement the estimation algorithm in a field-programmable gate array. The embedded package is shown to be a hardware/software co-design handling estimation procedure using finite precision arithmetic for high-speed computation. The accuracy of the finite precision FPGA hardware design is compared with the floating-point software evaluation of the algorithm on MATLAB to benchmark its performance and statistical consistency with the error measures. Implementation results demonstrate the utility of FPGA computing capabilities for high-speed proximity navigation using iVisNav.
Bias Reducing Multitask Learning on Mental Health Prediction
Venue: arXiv cs.LG 2022
Year: 2022

There has been an increase in research in developing machine learning models for mental health detection or prediction in recent years due to increased mental health issues in society. Effective use of mental health prediction or detection models can help mental health practitioners re-define mental illnesses more objectively than currently done, and identify illnesses at an earlier stage when interventions may be more effective. However, there is still a lack of standard in evaluating bias in such machine learning models in the field, which leads to challenges in providing reliable predictions and in addressing disparities. This lack of standards persists due to factors such as technical difficulties, complexities of high dimensional clinical health data, etc., which are especially true for physiological signals. This along with prior evidence of relations between some physiological signals with certain demographic identities restates the importance of exploring bias in mental health prediction models that utilize physiological signals. In this work, we aim to perform a fairness analysis and implement a multi-task learning based bias mitigation method on anxiety prediction models using ECG data. Our method is based on the idea of epistemic uncertainty and its relationship with model weights and feature space representation. Our analysis showed that our anxiety prediction base model introduced some bias with regards to age, income, ethnicity, and whether a participant is born in the U.S. or not, and our bias mitigation method performed better at reducing the bias in the model, when compared to the reweighting mitigation technique. Our analysis on feature importance also helped identify relationships between heart rate variability and multiple demographic groupings.
Generating Negative Samples for Sequential Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

To make Sequential Recommendation (SR) successful, recent works focus on designing effective sequential encoders, fusing side information, and mining extra positive self-supervision signals. The strategy of sampling negative items at each time step is less explored. Due to the dynamics of users' interests and model updates during training, considering randomly sampled items from a user's non-interacted item set as negatives can be uninformative. As a result, the model will inaccurately learn user preferences toward items. Identifying informative negatives is challenging because informative negative items are tied with both dynamically changed interests and model parameters (and sampling process should also be efficient). To this end, we propose to Generate Negative Samples (items) for SR (GenNi). A negative item is sampled at each time step based on the current SR model's learned user preferences toward items. An efficient implementation is proposed to further accelerate the generation process, making it scalable to large-scale recommendation tasks. Extensive experiments on four public datasets verify the importance of providing high-quality negative samples for SR and demonstrate the effectiveness and efficiency of GenNi.
A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
Venue: arXiv cs.LG 2022
Year: 2022

Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing overhead where inputs need to be zero-padded to the maximum sentence length in the batch to accommodate the parallel computing platforms. This paper targets the field-programmable gate array (FPGA) and proposes a coherent sequence length adaptive algorithm-hardware co-design for Transformer acceleration. Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm. The proposed sparse attention operator brings the complexity of attention-based models down to linear complexity and alleviates the off-chip memory traffic. The proposed length-aware resource hardware scheduling algorithm dynamically allocates the hardware resources to fill up the pipeline slots and eliminates bubbles for NLP tasks. Experiments show that our design has very small accuracy loss and has 80.2 $\times$ and 2.6 $\times$ speedup compared to CPU and GPU implementation, and 4 $\times$ higher energy efficiency than state-of-the-art GPU accelerator optimized via CUBLAS GEMM.
See What You See: Self-supervised Cross-modal Retrieval of Visual Stimuli from Brain Activity
Venue: arXiv cs.MM 2022
Year: 2022

Recent studies demonstrate the use of a two-stage supervised framework to generate images that depict human perception to visual stimuli from EEG, referring to EEG-visual reconstruction. They are, however, unable to reproduce the exact visual stimulus, since it is the human-specified annotation of images, not their data, that determines what the synthesized images are. Moreover, synthesized images often suffer from noisy EEG encodings and unstable training of generative models, making them hard to recognize. Instead, we present a single-stage EEG-visual retrieval paradigm where data of two modalities are correlated, as opposed to their annotations, allowing us to recover the exact visual stimulus for an EEG clip. We maximize the mutual information between the EEG encoding and associated visual stimulus through optimization of a contrastive self-supervised objective, leading to two additional benefits. One, it enables EEG encodings to handle visual classes beyond seen ones during training, since learning is not directed at class annotations. In addition, the model is no longer required to generate every detail of the visual stimulus, but rather focuses on cross-modal alignment and retrieves images at the instance level, ensuring distinguishable model output. Empirical studies are conducted on the largest single-subject EEG dataset that measures brain activities evoked by image stimuli. We demonstrate the proposed approach completes an instance-level EEG-visual retrieval task which existing methods cannot. We also examine the implications of a range of EEG and visual encoder structures. Furthermore, for a mostly studied semantic-level EEG-visual classification task, despite not using class annotations, the proposed method outperforms state-of-the-art supervised EEG-visual reconstruction approaches, particularly on the capability of open class recognition.
A New Robust Algorithm for Computation of a Triangle Circumscribed Sphere in E3 and a Hypersphere Simplex
Venue: arXiv cs.GR 2022
Year: 2022

There are many applications in which a bounding sphere containing the given triangle E3 is needed, e.g. fast collision detection, ray-triangle intersecting in raytracing etc. This is a typical geometrical problem in E3 and it has also applications in computational problems in general. In this paper a new fast and robust algorithm of circumscribed sphere computation in the -dimensional space is presented and specification for the E3 space is given, too. The presented method is convenient for use on GPU or with SSE or Intel AVX instructions on a standard CPU
Point-in-Convex Polygon and Point-in-Convex Polyhedron Algorithms with O(1) Complexity using Space Subdivision
Venue: arXiv cs.GR 2022
Year: 2022

There are many space subdivision and space partitioning techniques used in many algorithms to speed up computations. They mostly rely on orthogonal space subdivision, resp. using hierarchical data structures, e.g. BSP trees, quadtrees, octrees, kd-trees, bounding volume hierarchies, etc. However in some applications a non-orthogonal space subdivision can offer new ways for actual speed up. In the case of convex polygon in E3 a simple Point-in-Polygon test is of the O(N) complexity and the optimal algorithm is of O(lg N) computational complexity. In the E3 case, the complexity is O(N) even for the convex polyhedron as no ordering is defined. New Point-in-Convex Polygon and Point-in-Convex Polyhedron algorithms are presented based on space subdivision in the preprocessing stage resulting to O(1) run-time complexity. The presented approach is simple to implement. Due to the principle of duality, dual problems, e.g. line-convex polygon, line clipping, can be solved in a similarly.
Asynchronous scalable version of the Global-Local non-invasive coupling
Venue: arXiv cs.DC 2022
Year: 2022

The Global-Local non-invasive coupling is an improvement of the submodeling technique, which permits to locally enhance structure computations by introducing patches with refined models and to take into accounts all the interactions. In order to circumvent its inherently limited computational performance, we propose and implement an asynchronous version of the method. The asynchronous coupling reduces the dependency on communications, failures, and load imbalance. We present the theory and the implementation of the method in the linear case and illustrate its performance on academic cases inspired by actual industrial problems.
Data Leaves: Scenario-oriented Metadata for Data Federative Innovation
Venue: arXiv cs.DB 2022
Year: 2022

A method for representing the digest information of each dataset is proposed, oriented to the aid of innovative thoughts and the communication of data users who attempt to create valuable products, services, and business models using or combining datasets. Compared with methods for connecting datasets via shared attributes (i.e., variables), this method connects datasets via events, situations, or actions in a scenario that is supposed to be active in the real world. This method reflects the consideration of the fitness of each metadata to the feature concept, which is an abstract of the information or knowledge expected to be acquired from data; thus, the users of the data acquire practical knowledge that fits the requirements of real businesses and real life, as well as grounds for realistic application of AI technologies to data.
Multi-agent reinforcement learning for intent-based service assurance in cellular networks
Venue: arXiv cs.LG 2022
Year: 2022

Recently, intent-based management has received good attention in telecom networks owing to stringent performance requirements for many of the use cases. Several approaches in the literature employ traditional closed-loop driven methods to fulfill the intents on the KPIs. However, these methods consider every closed-loop independent of each other which degrades the combined performance. Also, such existing methods are not easily scalable. Multi-agent reinforcement learning (MARL) techniques have shown significant promise in many areas in which traditional closed-loop control falls short, typically for complex coordination and conflict management among loops. In this work, we propose a method based on MARL to achieve intent-based management without the need for knowing a model of the underlying system. Moreover, when there are conflicting intents, the MARL agents can implicitly incentivize the loops to cooperate and promote trade-offs, without human interaction, by prioritizing the important KPIs. Experiments have been performed on a network emulator for optimizing KPIs of three services. Results obtained demonstrate that the proposed system performs quite well and is able to fulfill all existing intents when there are enough resources or prioritize the KPIs when resources are scarce.
PS-NeRV: Patch-wise Stylized Neural Representations for Videos
Venue: arXiv cs.CV 2022
Year: 2022

We study how to represent a video with implicit neural representations (INRs). Classical INRs methods generally utilize MLPs to map input coordinates to output pixels. While some recent works have tried to directly reconstruct the whole image with CNNs. However, we argue that both the above pixel-wise and image-wise strategies are not favorable to video data. Instead, we propose a patch-wise solution, PS-NeRV, which represents videos as a function of patches and the corresponding patch coordinate. It naturally inherits the advantages of image-wise methods, and achieves excellent reconstruction performance with fast decoding speed. The whole method includes conventional modules, like positional embedding, MLPs and CNNs, while also introduces AdaIN to enhance intermediate features. These simple yet essential changes could help the network easily fit high-frequency details. Extensive experiments have demonstrated its effectiveness in several video-related tasks, such as video compression and video inpainting.
Learning Modular Structures That Generalize Out-of-Distribution
Venue: arXiv cs.LG 2022
Year: 2022

Out-of-distribution (O.O.D.) generalization remains to be a key challenge for real-world machine learning systems. We describe a method for O.O.D. generalization that, through training, encourages models to only preserve features in the network that are well reused across multiple training domains. Our method combines two complementary neuron-level regularizers with a probabilistic differentiable binary mask over the network, to extract a modular sub-network that achieves better O.O.D. performance than the original network. Preliminary evaluation on two benchmark datasets corroborates the promise of our method.
Adaptivity Gaps for the Stochastic Boolean Function Evaluation Problem
Venue: arXiv cs.DS 2022
Year: 2022

We consider the Stochastic Boolean Function Evaluation (SBFE) problem where the task is to efficiently evaluate a known Boolean function $f$ on an unknown bit string $x$ of length $n$. We determine $f(x)$ by sequentially testing the variables of $x$, each of which is associated with a cost of testing and an independent probability of being true. If a strategy for solving the problem is adaptive in the sense that its next test can depend on the outcomes of previous tests, it has lower expected cost but may take up to exponential space to store. In contrast, a non-adaptive strategy may have higher expected cost but can be stored in linear space and benefit from parallel resources. The adaptivity gap, the ratio between the expected cost of the optimal non-adaptive and adaptive strategies, is a measure of the benefit of adaptivity. We present lower bounds on the adaptivity gap for the SBFE problem for popular classes of Boolean functions, including read-once DNF formulas, read-once formulas, and general DNFs. Our bounds range from $Ω(\log n)$ to $Ω(n/\log n)$, contrasting with recent $O(1)$ gaps shown for symmetric functions and linear threshold functions.
When can I Speak? Predicting initiation points for spoken dialogue agents
Venue: arXiv cs.CL 2022
Year: 2022

Current spoken dialogue systems initiate their turns after a long period of silence (700-1000ms), which leads to little real-time feedback, sluggish responses, and an overall stilted conversational flow. Humans typically respond within 200ms and successfully predicting initiation points in advance would allow spoken dialogue agents to do the same. In this work, we predict the lead-time to initiation using prosodic features from a pre-trained speech representation model (wav2vec 1.0) operating on user audio and word features from a pre-trained language model (GPT-2) operating on incremental transcriptions. To evaluate errors, we propose two metrics w.r.t. predicted and true lead times. We train and evaluate the models on the Switchboard Corpus and find that our method outperforms features from prior work on both metrics and vastly outperforms the common approach of waiting for 700ms of silence.
Animated Vega-Lite: Unifying Animation with a Grammar of Interactive Graphics
Venue: arXiv cs.HC 2022
Year: 2022

We present Animated Vega-Lite, a set of extensions to Vega-Lite that model animated visualizations as time-varying data queries. In contrast to alternate approaches for specifying animated visualizations, which prize a highly expressive design space, Animated Vega-Lite prioritizes unifying animation with the language's existing abstractions for static and interactive visualizations to enable authors to smoothly move between or combine these modalities. Thus, to compose animation with static visualizations, we represent time as an encoding channel. Time encodings map a data field to animation keyframes, providing a lightweight specification for animations without interaction. To compose animation and interaction, we also represent time as an event stream; Vega-Lite selections, which provide dynamic data queries, are now driven not only by input events but by timer ticks as well. We evaluate the expressiveness of our approach through a gallery of diverse examples that demonstrate coverage over taxonomies of both interaction and animation. We also critically reflect on the conceptual affordances and limitations of our contribution by interviewing five expert developers of existing animation grammars. These reflections highlight the key motivating role of in-the-wild examples, and identify three central tradeoffs: the language design process, the types of animated transitions supported, and how the systems model keyframes.