OpenRaiser/Intern-Atlas huggingface dataset:
Scale up your In-Memory Accelerator: Leveraging Wireless-on-Chip Communication for AIMC-based CNN Inference
Venue: arXiv cs.AR 2022
Year: 2022

Analog In-Memory Computing (AIMC) is emerging as a disruptive paradigm for heterogeneous computing, potentially delivering orders of magnitude better peak performance and efficiency over traditional digital signal processing architectures on Matrix-Vector multiplication. However, to sustain this throughput in real-world applications, AIMC tiles must be supplied with data at very high bandwidth and low latency; this poses an unprecedented pressure on the on-chip communication infrastructure, which becomes the system's performance and efficiency bottleneck. In this context, the performance and plasticity of emerging on-chip wireless communication paradigms provide the required breakthrough to up-scale on-chip communication in large AIMC devices. This work presents a many-tile AIMC architecture with inter-tile wireless communication that integrates multiple heterogeneous computing clusters, embedding a mix of parallel RISC-V cores and AIMC tiles. We perform an extensive design space exploration of the proposed architecture and discuss the benefits of exploiting emerging on-chip communication technologies such as wireless transceivers in the millimeter-wave and terahertz bands.
Explainable Artificial Intelligence (XAI) for Internet of Things: A Survey
Venue: arXiv cs.AI 2022
Year: 2022

Black-box nature of Artificial Intelligence (AI) models do not allow users to comprehend and sometimes trust the output created by such model. In AI applications, where not only the results but also the decision paths to the results are critical, such black-box AI models are not sufficient. Explainable Artificial Intelligence (XAI) addresses this problem and defines a set of AI models that are interpretable by the users. Recently, several number of XAI models have been to address the issues surrounding by lack of interpretability and explainability of black-box models in various application areas such as healthcare, military, energy, financial and industrial domains. Although the concept of XAI has gained great deal of attention recently, its integration into the IoT domain has not yet been fully defined. In this paper, we provide an in-depth and systematic review of recent studies using XAI models in the scope of IoT domain. We categorize the studies according to their methodology and applications areas. In addition, we aim to focus on the challenging problems and open issues and give future directions to guide the developers and researchers for prospective future investigations.
Learning Attention-based Representations from Multiple Patterns for Relation Prediction in Knowledge Graphs
Venue: arXiv cs.AI 2022
Year: 2022

Knowledge bases, and their representations in the form of knowledge graphs (KGs), are naturally incomplete. Since scientific and industrial applications have extensively adopted them, there is a high demand for solutions that complete their information. Several recent works tackle this challenge by learning embeddings for entities and relations, then employing them to predict new relations among the entities. Despite their aggrandizement, most of those methods focus only on the local neighbors of a relation to learn the embeddings. As a result, they may fail to capture the KGs' context information by neglecting long-term dependencies and the propagation of entities' semantics. In this manuscript, we propose ÆMP (Attention-based Embeddings from Multiple Patterns), a novel model for learning contextualized representations by: (i) acquiring entities' context information through an attention-enhanced message-passing scheme, which captures the entities' local semantics while focusing on different aspects of their neighborhood; and (ii) capturing the semantic context, by leveraging the paths and their relationships between entities. Our empirical findings draw insights into how attention mechanisms can improve entities' context representation and how combining entities and semantic path contexts improves the general representation of entities and the relation predictions. Experimental results on several large and small knowledge graph benchmarks show that ÆMP either outperforms or competes with state-of-the-art relation prediction methods.
Robot Control for Simultaneous Impact Tasks through Time-Invariant Reference Spreading
Venue: arXiv cs.RO 2022
Year: 2022

With the goal of enabling the exploitation of impacts in robotic manipulation, a new framework is presented for control of robotic manipulators that are tasked to execute nominally simultaneous impacts. In this framework, we employ tracking of time-invariant reference vector fields corresponding to the ante- and post-impact motion, increasing its applicability over similar conventional tracking control approaches. The ante- and post-impact references are coupled through a rigid impact map, and are extended to overlap around the area where the impact is expected to take place, such that the reference corresponding to the actual contact state of the robot can always be followed. As a sequence of impacts at the different contact points will typically occur, resulting in uncertainty of the contact mode and unreliable velocity measurements, a new interim control mode catered towards time-invariant references is formulated. In this mode, a position feedback signal is derived from the ante-impact velocity reference, which is used to enforce sustained contact in all contact points without using velocity feedback. With an eye towards real implementation, the approach is formulated using a QP control framework, and is validated using numerical simulations both on a rigid robot with a hard inelastic contact model and on a realistic robot model with flexible joints and compliant partially elastic contact model.
Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement
Venue: arXiv cs.CV 2022
Year: 2022

Heterogeneous Face Recognition (HFR) aims to match faces across different domains (e.g., visible to near-infrared images), which has been widely applied in authentication and forensics scenarios. However, HFR is a challenging problem because of the large cross-domain discrepancy, limited heterogeneous data pairs, and large variation of facial attributes. To address these challenges, we propose a new HFR method from the perspective of heterogeneous data augmentation, named Face Synthesis with Identity-Attribute Disentanglement (FSIAD). Firstly, the identity-attribute disentanglement (IAD) decouples face images into identity-related representations and identity-unrelated representations (called attributes), and then decreases the correlation between identities and attributes. Secondly, we devise a face synthesis module (FSM) to generate a large number of images with stochastic combinations of disentangled identities and attributes for enriching the attribute diversity of synthetic images. Both the original images and the synthetic ones are utilized to train the HFR network for tackling the challenges and improving the performance of HFR. Extensive experiments on five HFR databases validate that FSIAD obtains superior performance than previous HFR approaches. Particularly, FSIAD obtains 4.8% improvement over state of the art in terms of VR@FAR=0.01% on LAMP-HQ, the largest HFR database so far.
A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation
Venue: arXiv cs.CL 2022
Year: 2022

Chinese dialects are different variations of Chinese and can be considered as different languages in the same language family with Mandarin. Though they all use Chinese characters, the pronunciations, grammar and idioms can vary significantly, and even local speakers may find it hard to input correct written forms of dialect. Besides, using Mandarin text as text-to-speech inputs would generate speech with poor naturalness. In this paper, we propose a novel Chinese dialect TTS frontend with a translation module, which converts Mandarin text into dialectic expressions to improve the intelligibility and naturalness of synthesized speech. A non-autoregressive neural machine translation model with various tricks is proposed for the translation task. It is the first known work to incorporate translation with TTS frontend. Experiments on Cantonese show the proposed model improves 2.56 BLEU and TTS improves 0.27 MOS with Mandarin inputs.
Evolutionary Echo State Network: evolving reservoirs in the Fourier space
Venue: arXiv cs.NE 2022
Year: 2022

The Echo State Network (ESN) is a class of Recurrent Neural Network with a large number of hidden-hidden weights (in the so-called reservoir). Canonical ESN and its variations have recently received significant attention due to their remarkable success in the modeling of non-linear dynamical systems. The reservoir is randomly connected with fixed weights that don't change in the learning process. Only the weights from reservoir to output are trained. Since the reservoir is fixed during the training procedure, we may wonder if the computational power of the recurrent structure is fully harnessed. In this article, we propose a new computational model of the ESN type, that represents the reservoir weights in the Fourier space and performs a fine-tuning of these weights applying genetic algorithms in the frequency domain. The main interest is that this procedure will work in a much smaller space compared to the classical ESN, thus providing a dimensionality reduction transformation of the initial method. The proposed technique allows us to exploit the benefits of the large recurrent structure avoiding the training problems of gradient-based method. We provide a detailed experimental study that demonstrates the good performances of our approach with well-known chaotic systems and real-world data.
Weighted Ensembles for Active Learning with Adaptivity
Venue: arXiv cs.LG 2022
Year: 2022

Labeled data can be expensive to acquire in several application domains, including medical imaging, robotics, and computer vision. To efficiently train machine learning models under such high labeling costs, active learning (AL) judiciously selects the most informative data instances to label on-the-fly. This active sampling process can benefit from a statistical function model, that is typically captured by a Gaussian process (GP). While most GP-based AL approaches rely on a single kernel function, the present contribution advocates an ensemble of GP models with weights adapted to the labeled data collected incrementally. Building on this novel EGP model, a suite of acquisition functions emerges based on the uncertainty and disagreement rules. An adaptively weighted ensemble of EGP-based acquisition functions is also introduced to further robustify performance. Extensive tests on synthetic and real datasets showcase the merits of the proposed EGP-based approaches with respect to the single GP-based AL alternatives.
Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities
Venue: arXiv cs.CL 2022
Year: 2022

Empathy is a vital factor that contributes to mutual understanding, and joint problem-solving. In recent years, a growing number of studies have recognized the benefits of empathy and started to incorporate empathy in conversational systems. We refer to this topic as empathetic conversational systems. To identify the critical gaps and future opportunities in this topic, this paper examines this rapidly growing field using five review dimensions: (i) conceptual empathy models and frameworks, (ii) adopted empathy-related concepts, (iii) datasets and algorithmic techniques developed, (iv) evaluation strategies, and (v) state-of-the-art approaches. The findings show that most studies have centered on the use of the EMPATHETICDIALOGUES dataset, and the text-based modality dominates research in this field. Studies mainly focused on extracting features from the messages of the users and the conversational systems, with minimal emphasis on user modeling and profiling. Notably, studies that have incorporated emotion causes, external knowledge, and affect matching in the response generation models, have obtained significantly better results. For implementation in diverse real-world settings, we recommend that future studies should address key gaps in areas of detecting and authenticating emotions at the entity level, handling multimodal inputs, displaying more nuanced empathetic behaviors, and encompassing additional dialogue system features.
Comparative study in fair division algorithms
Venue: arXiv cs.GT 2022
Year: 2022

A comparison of four fair division algorithms performed on real data from the spliddit website. The comparison was made on the sum of agent's utilities, and the minimum utility for an agent in an allocation.
Explanation as Question Answering based on a Task Model of the Agent's Design
Venue: arXiv cs.HC 2022
Year: 2022

We describe a stance towards the generation of explanations in AI agents that is both human-centered and design-based. We collect questions about the working of an AI agent through participatory design by focus groups. We capture an agent's design through a Task-Method-Knowledge model that explicitly specifies the agent's tasks and goals, as well as the mechanisms, knowledge and vocabulary it uses for accomplishing the tasks. We illustrate our approach through the generation of explanations in Skillsync, an AI agent that links companies and colleges for worker upskilling and reskilling. In particular, we embed a question-answering agent called AskJill in Skillsync, where AskJill contains a TMK model of Skillsync's design. AskJill presently answers human-generated questions about Skillsync's tasks and vocabulary, and thereby helps explain how it produces its recommendations.
The Construction and Evaluation of the LEAFTOP Dataset of Automatically Extracted Nouns in 1480 Languages
Venue: arXiv cs.CL 2022
Year: 2022

The LEAFTOP (language extracted automatically from thousands of passages) dataset consists of nouns that appear in multiple places in the four gospels of the New Testament. We use a naive approach -- probabilistic inference -- to identify likely translations in 1480 other languages. We evaluate this process and find that it provides lexiconaries with accuracy from 42% (Korafe) to 99% (Runyankole), averaging 72% correct across evaluated languages. The process translates up to 161 distinct lemmas from Koine Greek (average 159). We identify nouns which appear to be easy and hard to translate, language families where this technique works, and future possible improvements and extensions. The claims to novelty are: the use of a Koine Greek New Testament as the source language; using a fully-annotated manually-created grammatically parse of the source text; a custom scraper for texts in the target languages; a new metric for language similarity; a novel strategy for evaluation on low-resource languages.
Meta-data Study in Autism Spectrum Disorder Classification Based on Structural MRI
Venue: arXiv cs.LG 2022
Year: 2022

Accurate diagnosis of autism spectrum disorder (ASD) based on neuroimaging data has significant implications, as extracting useful information from neuroimaging data for ASD detection is challenging. Even though machine learning techniques have been leveraged to improve the information extraction from neuroimaging data, the varying data quality caused by different meta-data conditions (i.e., data collection strategies) limits the effective information that can be extracted, thus leading to data-dependent predictive accuracies in ASD detection, which can be worse than random guess in some cases. In this work, we systematically investigate the impact of three kinds of meta-data on the predictive accuracy of classifying ASD based on structural MRI collected from 20 different sites, where meta-data conditions vary.
State complexity of the star of a Boolean operation
Venue: arXiv cs.FL 2022
Year: 2022

Monsters and modifiers are two concepts recently developed in the state complexity theory. A monster is an automaton in which every function from states to states is represented by at least one letter. A modifier is a set of functions allowing one to transform a set of automata into one automaton. The paper describes a general strategy that can be used to compute the state complexity of many operations. We illustrate it on the problem of the star of a Boolean operation. After applying modifiers on monsters, the states of the resulting automata are assimilated to combinatorial objects: the tableaux. We investigate the combinatorics of these tableaux in order to deduce the state complexity. Specifically, we recover the state complexity of star of intersection and star of union, and we also give the exact state complexity of star of symmetrical difference. We thus harmonize the search strategy for the state complexity of star of any Boolean operations.
Real-time Hyper-Dimensional Reconfiguration at the Edge using Hardware Accelerators
Venue: arXiv cs.CV 2022
Year: 2022

In this paper we present Hyper-Dimensional Reconfigurable Analytics at the Tactical Edge (HyDRATE) using low-SWaP embedded hardware that can perform real-time reconfiguration at the edge leveraging non-MAC (free of floating-point MultiplyACcumulate operations) deep neural nets (DNN) combined with hyperdimensional (HD) computing accelerators. We describe the algorithm, trained quantized model generation, and simulated performance of a feature extractor free of multiply-accumulates feeding a hyperdimensional logic-based classifier. Then we show how performance increases with the number of hyperdimensions. We describe the realized low-SWaP FPGA hardware and embedded software system compared to traditional DNNs and detail the implemented hardware accelerators. We discuss the measured system latency and power, noise robustness due to use of learnable quantization and HD computing, actual versus simulated system performance for a video activity classification task and demonstration of reconfiguration on this same dataset. We show that reconfigurability in the field is achieved by retraining only the feed-forward HD classifier without gradient descent backpropagation (gradient-free), using few-shot learning of new classes at the edge. Initial work performed used LRCN DNN and is currently extended to use Two-stream DNN with improved performance.
Finite electro-elasticity with physics-augmented neural networks
Venue: arXiv cs.CE 2022
Year: 2022

In the present work, a machine learning based constitutive model for electro-mechanically coupled material behavior at finite deformations is proposed. Using different sets of invariants as inputs, an internal energy density is formulated as a convex neural network. In this way, the model fulfills the polyconvexity condition which ensures material stability, as well as thermodynamic consistency, objectivity, material symmetry, and growth conditions. Depending on the considered invariants, this physics-augmented machine learning model can either be applied for compressible or nearly incompressible material behavior, as well as for arbitrary material symmetry classes. The applicability and versatility of the approach is demonstrated by calibrating it on transversely isotropic data generated with an analytical potential, as well as for the effective constitutive modeling of an analytically homogenized, transversely isotropic rank-one laminate composite and a numerically homogenized cubic metamaterial. These examinations show the excellent generalization properties that physics-augmented neural networks offer also for multi-physical material modeling such as nonlinear electro-elasticity.
Multifidelity Reinforcement Learning with Control Variates
Venue: arXiv cs.LG 2022
Year: 2022

In many computational science and engineering applications, the output of a system of interest corresponding to a given input can be queried at different levels of fidelity with different costs. Typically, low-fidelity data is cheap and abundant, while high-fidelity data is expensive and scarce. In this work we study the reinforcement learning (RL) problem in the presence of multiple environments with different levels of fidelity for a given control task. We focus on improving the RL agent's performance with multifidelity data. Specifically, a multifidelity estimator that exploits the cross-correlations between the low- and high-fidelity returns is proposed to reduce the variance in the estimation of the state-action value function. The proposed estimator, which is based on the method of control variates, is used to design a multifidelity Monte Carlo RL (MFMCRL) algorithm that improves the learning of the agent in the high-fidelity environment. The impacts of variance reduction on policy evaluation and policy improvement are theoretically analyzed by using probability bounds. Our theoretical analysis and numerical experiments demonstrate that for a finite budget of high-fidelity data samples, our proposed MFMCRL agent attains superior performance compared with that of a standard RL agent that uses only the high-fidelity environment data for learning the optimal policy.
Reducing the local alphabet size in tiling systems by means of 2D comma-free codes
Venue: arXiv cs.FL 2022
Year: 2022

A recognizable picture language is defined as the projection of a local picture language defined by a set of two-by-two tiles, i.e. by a strictly-locally-testable (SLT) language of order 2. The family of recognizable picture languages is also defined, using larger $k$ by $k$ tiles, $k>2$, by the projection of the corresponding SLT language. A basic measure of the descriptive complexity of a picture language is given by the size of the SLT alphabet using two-by-two tiles, more precisely by the so-called alphabetic ratio of sizes: SLT-alphabet / picture-alphabet. We study how the alphabetic ratio changes moving from tiles of size two to tiles of larger size, and we obtain the following result: any recognizable picture language over an alphabet of size $n$ is the projection of an SLT language over an alphabet of size $2n$. Moreover, two is the minimal alphabetic ratio possible in general. The proof relies on a new family of comma-free picture codes, for which a lower bound on numerosity is established; and on the relation of languages of encoded pictures with SLT languages. Our result reproduces in two dimensions a similar property (known as Extended Medvedev's theorem) of the regular word languages, concerning the minimal alphabetic ratio needed to define a language by means of a projection of an SLT word language.
Explaining Image Classifiers Using Contrastive Counterfactuals in Generative Latent Spaces
Venue: arXiv cs.CV 2022
Year: 2022

Despite their high accuracies, modern complex image classifiers cannot be trusted for sensitive tasks due to their unknown decision-making process and potential biases. Counterfactual explanations are very effective in providing transparency for these black-box algorithms. Nevertheless, generating counterfactuals that can have a consistent impact on classifier outputs and yet expose interpretable feature changes is a very challenging task. We introduce a novel method to generate causal and yet interpretable counterfactual explanations for image classifiers using pretrained generative models without any re-training or conditioning. The generative models in this technique are not bound to be trained on the same data as the target classifier. We use this framework to obtain contrastive and causal sufficiency and necessity scores as global explanations for black-box classifiers. On the task of face attribute classification, we show how different attributes influence the classifier output by providing both causal and contrastive feature attributions, and the corresponding counterfactual images.