OpenRaiser/Intern-Atlas huggingface dataset:
Energy Constraints Improve Liquid State Machine Performance
Venue: arXiv cs.NE 2020
Year: 2020

A model of metabolic energy constraints is applied to a liquid state machine in order to analyze its effects on network performance. It was found that, in certain combinations of energy constraints, a significant increase in testing accuracy emerged; an improvement of 4.25% was observed on a seizure detection task using a digital liquid state machine while reducing overall reservoir spiking activity by 6.9%. The accuracy improvements appear to be linked to the energy constraints' impact on the reservoir's dynamics, as measured through metrics such as the Lyapunov exponent and the separation of the reservoir.
Nonlinear Higher-Order Label Spreading
Venue: arXiv cs.LG 2020
Year: 2020

Label spreading is a general technique for semi-supervised learning with point cloud or network data, which can be interpreted as a diffusion of labels on a graph. While there are many variants of label spreading, nearly all of them are linear models, where the incoming information to a node is a weighted sum of information from neighboring nodes. Here, we add nonlinearity to label spreading through nonlinear functions of higher-order structure in the graph, namely triangles in the graph. For a broad class of nonlinear functions, we prove convergence of our nonlinear higher-order label spreading algorithm to the global solution of a constrained semi-supervised loss function. We demonstrate the efficiency and efficacy of our approach on a variety of point cloud and network datasets, where the nonlinear higher-order model compares favorably to classical label spreading, as well as hypergraph models and graph neural networks.
A Heuristically Self-Organised Linguistic Attribute Deep Learning in Edge Computing For IoT Intelligence
Venue: arXiv cs.AI 2020
Year: 2020

With the development of Internet of Things (IoT), IoT intelligence becomes emerging technology. "Curse of Dimensionality" is the barrier of data fusion in edge devices for the success of IoT intelligence. A Linguistic Attribute Hierarchy (LAH), embedded with Linguistic Decision Trees (LDTs), can represent a new attribute deep learning. In contrast to the conventional deep learning, an LAH could overcome the shortcoming of missing interpretation by providing transparent information propagation through the rules, produced by LDTs in the LAH. Similar to the conventional deep learning, the computing complexity of optimising LAHs blocks the applications of LAHs. In this paper, we propose a heuristic approach to constructing an LAH, embedded with LDTs for decision making or classification by utilising the distance correlations between attributes and between attributes and the goal variable. The set of attributes is divided to some attribute clusters, and then they are heuristically organised to form a linguistic attribute hierarchy. The proposed approach was validated with some benchmark decision making or classification problems from the UCI machine learning repository. The experimental results show that the proposed self-organisation algorithm can construct an effective and efficient linguistic attribute hierarchy. Such a self-organised linguistic attribute hierarchy embedded with LDTs can not only efficiently tackle "curse of dimensionality" in a single LDT for data fusion with massive attributes, but also achieve better or comparable performance on decision making or classification, compared to the single LDT for the problem to be solved. The self-organisation algorithm is much efficient than the Genetic Algorithm in Wrapper for the optimisation of LAHs. This makes it feasible to embed the self-organisation algorithm in edge devices for IoT intelligence.
tvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder
Venue: arXiv cs.LG 2020
Year: 2020

Variational autoencoders (VAEs) are a powerful class of deep generative latent variable model for unsupervised representation learning on high-dimensional data. To ensure computational tractability, VAEs are often implemented with a univariate standard Gaussian prior and a mean-field Gaussian variational posterior distribution. This results in a vector-valued latent variables that are agnostic to the original data structure which might be highly correlated across and within multiple dimensions. We propose a tensor-variate extension to the VAE framework, the tensor-variate Gaussian process prior variational autoencoder (tvGP-VAE), which replaces the standard univariate Gaussian prior and posterior distributions with tensor-variate Gaussian processes. The tvGP-VAE is able to explicitly model correlation structures via the use of kernel functions over the dimensions of tensor-valued latent variables. Using spatiotemporally correlated image time series as an example, we show that the choice of which correlation structures to explicitly represent in the latent space has a significant impact on model performance in terms of reconstruction.
Yield Loss Reduction and Test of AI and Deep Learning Accelerators
Venue: arXiv cs.DC 2020
Year: 2020

With data-driven analytics becoming mainstream, the global demand for dedicated AI and Deep Learning accelerator chips is soaring. These accelerators, designed with densely packed Processing Elements (PE), are especially vulnerable to the manufacturing defects and functional faults common in the advanced semiconductor process nodes resulting in significant yield loss. In this work, we demonstrate an application-driven methodology of binning the AI accelerator chips, and yield loss reduction by correlating the circuit faults in the PEs of the accelerator with the desired accuracy of the target AI workload. We exploit the inherent fault tolerance features of trained deep learning models and a strategy of selective deactivation of faulty PEs to develop the presented yield loss reduction and test methodology. An analytical relationship is derived between fault location, fault rate, and the AI task's accuracy for deciding if the accelerator chip can pass the final yield test. A yield-loss reduction aware fault isolation, ATPG, and test flow are presented for the multiply and accumulate units of the PEs. Results obtained with widely used AI/deep learning benchmarks demonstrate that the accelerators can sustain 5% fault-rate in PE arrays while suffering from less than 1% accuracy loss, thus enabling product-binning and yield loss reduction of these chips.
Modeling Long-horizon Tasks as Sequential Interaction Landscapes
Venue: arXiv cs.RO 2020
Year: 2020

Complex object manipulation tasks often span over long sequences of operations. Task planning over long-time horizons is a challenging and open problem in robotics, and its complexity grows exponentially with an increasing number of subtasks. In this paper we present a deep learning network that learns dependencies and transitions across subtasks solely from a set of demonstration videos. We represent each subtask as an action symbol (e.g. move cup), and show that these symbols can be learned and predicted directly from image observations. Learning from demonstrations and visual observations are two main pillars of our approach. The former makes the learning tractable as it provides the network with information about the most frequent transitions and relevant dependency between subtasks (instead of exploring all possible combination), while the latter allows the network to continuously monitor the task progress and thus to interactively adapt to changes in the environment. We evaluate our framework on two long horizon tasks: (1) block stacking of puzzle pieces being executed by humans, and (2) a robot manipulation task involving pick and place of objects and sliding a cabinet door with a 7-DoF robot arm. We show that complex plans can be carried out when executing the robotic task and the robot can interactively adapt to changes in the environment and recover from failure cases.
Integer Programming for Multi-Robot Planning: A Column Generation Approach
Venue: arXiv cs.AI 2020
Year: 2020

We consider the problem of coordinating a fleet of robots in a warehouse so as to maximize the reward achieved within a time limit while respecting problem and robot specific constraints. We formulate the problem as a weighted set packing problem where elements are defined as being the space-time positions a robot can occupy and the items that can be picked up and delivered. We enforce that robots do not collide, that each item is delivered at most once, and that the number of robots active at any time does not exceed the total number available. Since the set of robot routes is not enumerable, we attack optimization using column generation where pricing is a resource-constrained shortest-path problem.
How Smart is the Grid?
Venue: arXiv cs.CY 2020
Year: 2020

Ancient Romans called 'urbs' the set of buildings and infrastructures, and 'civitas' the Roman citizens. Today instead, while the society is surfing the digital tsunami, 'urbs' and 'civitas' tend to become much closer, almost merging, that we might attempt to condensate these into a single concept: 'smart grid'. Internet of things, artificial intelligence, blockchain, quantum cryptography is only a few of the paradigms that are likely to contribute to determining the final portrait of the future smart grid. However, to understand the effective sustainability of complex grids, specific tools are required. To this end, in this article, a systematic review of the emerging paradigms is presented, identifying intersectoral synergies and limitations with respect to the `smart grid' concept. Further, a taxonomic framework for assessing the level of sustainability of the grid is proposed. Finally, from the scenario portrayed, a set of issues involving engineering, regulation, security, and social frameworks have been derived in a theoretical fashion. The findings are likely to suggest the urgent need for multidisciplinary cooperation to wisely address engineering and ontological challenges gravitating around the smart grid concept.
Gradient Boosting Application in Forecasting of Performance Indicators Values for Measuring the Efficiency of Promotions in FMCG Retail
Venue: arXiv cs.CY 2020
Year: 2020

In the paper, a problem of forecasting promotion efficiency is raised. The authors propose a new approach, using the gradient boosting method for this task. Six performance indicators are introduced to capture the promotion effect. For each of them, within predefined groups of products, a model was trained. A description of using these models for forecasting and optimising promotion efficiency is provided. Data preparation and hyperparameters tuning processes are also described. The experiments were performed for three groups of products from a large grocery company.
HausaMT v1.0: Towards English-Hausa Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

Neural Machine Translation (NMT) for low-resource languages suffers from low performance because of the lack of large amounts of parallel data and language diversity. To contribute to ameliorating this problem, we built a baseline model for English-Hausa machine translation, which is considered a task for low-resource language. The Hausa language is the second largest Afro-Asiatic language in the world after Arabic and it is the third largest language for trading across a larger swath of West Africa countries, after English and French. In this paper, we curated different datasets containing Hausa-English parallel corpus for our translation. We trained baseline models and evaluated the performance of our models using the Recurrent and Transformer encoder-decoder architecture with two tokenization approaches: standard word-level tokenization and Byte Pair Encoding (BPE) subword tokenization.
Folding Simulation of Rigid Origami with Lagrange Multiplier Method
Venue: arXiv cs.CE 2020
Year: 2020

Origami crease patterns are folding paths that transform flat sheets into spatial objects. Origami patterns with a single degree of freedom (DOF) have creases that fold simultaneously. More often, several substeps are required to sequentially fold origami of multiple DOFs, and at each substep some creases fold and the rest remain fixed. In this study, we combine the loop closure constraint with Lagrange multiplier method to account for the sequential folding of rigid origami of multiple DOFs, by controlling the rotation of different sets of creases during successive substeps. This strategy is also applicable to model origami-inspired devices, where creases may be equipped with rotational springs and the folding process involves elastic energy. Several examples are presented to verify the proposed algorithms in tracing the sequential folding process as well as searching the equilibrium configurations of origami with rotational springs.
Learning Navigation Costs from Demonstration with Semantic Observations
Venue: arXiv cs.LG 2020
Year: 2020

This paper focuses on inverse reinforcement learning (IRL) for autonomous robot navigation using semantic observations. The objective is to infer a cost function that explains demonstrated behavior while relying only on the expert's observations and state-control trajectory. We develop a map encoder, which infers semantic class probabilities from the observation sequence, and a cost encoder, defined as deep neural network over the semantic features. Since the expert cost is not directly observable, the representation parameters can only be optimized by differentiating the error between demonstrated controls and a control policy computed from the cost estimate. The error is optimized using a closed-form subgradient computed only over a subset of promising states via a motion planning algorithm. We show that our approach learns to follow traffic rules in the autonomous driving CARLA simulator by relying on semantic observations of cars, sidewalks and road lanes.
Policy-focused Agent-based Modeling using RL Behavioral Models
Venue: arXiv cs.LG 2020
Year: 2020

Agent-based Models (ABMs) are valuable tools for policy analysis. ABMs help analysts explore the emergent consequences of policy interventions in multi-agent decision-making settings. But the validity of inferences drawn from ABM explorations depends on the quality of the ABM agents' behavioral models. Standard specifications of agent behavioral models rely either on heuristic decision-making rules or on regressions trained on past data. Both prior specification modes have limitations. This paper examines the value of reinforcement learning (RL) models as adaptive, high-performing, and behaviorally-valid models of agent decision-making in ABMs. We test the hypothesis that RL agents are effective as utility-maximizing agents in policy ABMs. We also address the problem of adapting RL algorithms to handle multi-agency in games by adapting and extending methods from recent literature. We evaluate the performance of such RL-based ABM agents via experiments on two policy-relevant ABMs: a minority game ABM, and an ABM of Influenza Transmission. We run some analytic experiments on our AI-equipped ABMs e.g. explorations of the effects of behavioral heterogeneity in a population and the emergence of synchronization in a population. The experiments show that RL behavioral models are effective at producing reward-seeking or reward-maximizing behaviors in ABM agents. Furthermore, RL behavioral models can learn to outperform the default adaptive behavioral models in the two ABMs examined.
Dynamic Interleaving of Content and Structure for Robust Indexing of Semi-Structured Hierarchical Data (Extended Version)
Venue: arXiv cs.DB 2020
Year: 2020

We propose a robust index for semi-structured hierarchical data that supports content-and-structure (CAS) queries specified by path and value predicates. At the heart of our approach is a novel dynamic interleaving scheme that merges the path and value dimensions of composite keys in a balanced way. We store these keys in our trie-based Robust Content-And-Structure index, which efficiently supports a wide range of CAS queries, including queries with wildcards and descendant axes. Additionally, we show important properties of our scheme, such as robustness against varying selectivities, and demonstrate improvements of up to two orders of magnitude over existing approaches in our experimental evaluation.
Smooth Proxy-Anchor Loss for Noisy Metric Learning
Venue: arXiv cs.CV 2020
Year: 2020

Many industrial applications use Metric Learning as a way to circumvent scalability issues when designing systems with a high number of classes. Because of this, this field of research is attracting a lot of interest from the academic and non-academic communities. Such industrial applications require large-scale datasets, which are usually generated with web data and, as a result, often contain a high number of noisy labels. While Metric Learning systems are sensitive to noisy labels, this is usually not tackled in the literature, that relies on manually annotated datasets. In this work, we propose a Metric Learning method that is able to overcome the presence of noisy labels using our novel Smooth Proxy-Anchor Loss. We also present an architecture that uses the aforementioned loss with a two-phase learning procedure. First, we train a confidence module that computes sample class confidences. Second, these confidences are used to weight the influence of each sample for the training of the embeddings. This results in a system that is able to provide robust sample embeddings. We compare the performance of the described method with current state-of-the-art Metric Learning losses (proxy-based and pair-based), when trained with a dataset containing noisy labels. The results showcase an improvement of 2.63 and 3.29 in Recall@1 with respect to MultiSimilarity and Proxy-Anchor Loss respectively, proving that our method outperforms the state-of-the-art of Metric Learning in noisy labeling conditions.
Eliciting Touristic Profiles: A User Study on Picture Collections
Venue: arXiv cs.HC 2020
Year: 2020

Eliciting the preferences and needs of tourists is challenging, since people often have difficulties to explicitly express them, especially in the initial phase of travel planning. Recommender systems employed at the early stage of planning can therefore be very beneficial to the general satisfaction of a user. Previous studies have explored pictures as a tool of communication and as a way to implicitly deduce a traveller's preferences and needs. In this paper, we conduct a user study to verify previous claims and conceptual work on the feasibility of modelling travel interests from a selection of a user's pictures. We utilize fine-tuned convolutional neural networks to compute a vector representation of a picture, where each dimension corresponds to a travel behavioural pattern from the traditional Seven-Factor model. In our study, we followed strict privacy principles and did not save uploaded pictures after computing their vector representation. We aggregate the representations of the pictures of a user into a single user representation, i.e., touristic profile, using different strategies. In our user study with 81 participants, we let users adjust the predicted touristic profile and confirm the usefulness of our approach. Our results show that given a collection of pictures the touristic profile of a user can be determined.
Re-evaluating phoneme frequencies
Venue: arXiv cs.CL 2020
Year: 2020

Causal processes can give rise to distinctive distributions in the linguistic variables that they affect. Consequently, a secure understanding of a variable's distribution can hold a key to understanding the forces that have causally shaped it. A storied distribution in linguistics has been Zipf's law, a kind of power law. In the wake of a major debate in the sciences around power-law hypotheses and the unreliability of earlier methods of evaluating them, here we re-evaluate the distributions claimed to characterize phoneme frequencies. We infer the fit of power laws and three alternative distributions to 166 Australian languages, using a maximum likelihood framework. We find evidence supporting earlier results, but also nuancing them and increasing our understanding of them. Most notably, phonemic inventories appear to have a Zipfian-like frequency structure among their most-frequent members (though perhaps also a lognormal structure) but a geometric (or exponential) structure among the least-frequent. We compare these new insights the kinds of causal processes that affect the evolution of phonemic inventories over time, and identify a potential account for why, despite there being an important role for phonetic substance in phonemic change, we could still expect inventories with highly diverse phonetic content to share similar distributions of phoneme frequencies. We conclude with priorities for future work in this promising program of research.
Beamwidth Selection for a Uniform Planar Array (UPA) Using RT-ICM mmWave Clusters
Venue: arXiv cs.IT 2020
Year: 2020

Beamforming is the primary technology to overcome the high path loss in millimeter-wave (mmWave) channels. Hence, performance improvement needs knowledge and control of the spatial domain. In particular, antenna structure and radiation parameters affect the beamforming performance in mmWave communications systems. In order to address the impairments such as beam misalignments, outage loss, tracking inability, blockage, etc., an optimum value of the beamwidth must be determined. In our previous paper, assuming a communication system that creates a beam per cluster, we theoretically investigated the beamwidth-received power relation in the cluster level mmWave channels. We used uniform linear array (ULA) antenna in our analysis. In this paper, we revisit the analysis and update the expressions for the scenario where we use rectangular uniform planar array (R-UPA) antenna. Rectangular beam model is considered to approximate the main lobe pattern of the antenna. For the channel, we derive beamwidth-dependent extracted power expressions for two intra-cluster channel models, IEEE 802.11ad and our previous work based on ray-tracing (RT-ICM). Combining antenna and channel gains, in case of the perfect alignment, we confirm that the optimum beamwidth converges zero. Performing asymptotic analysis of the received power, we give the formulation and insights that the practical nonzero beamwidth values can be achieved although sacrificing subtle from the maximum received power. Our analysis shows that to reach 95% of the maximum power for a typical indoor mmWave cluster, a practical beamwidth of 3.5 deg is enough. Finally, our analysis results show that there is a 13 dB increase in the maximum theoretical received power when UPA is used over ULA. We show that an 8 x 8 UPA can reach 50% of that maximum received power while the received power is still 10 dB larger than the ULA scenario.
An Overview of Deep Semi-Supervised Learning
Venue: arXiv cs.LG 2020
Year: 2020

Deep neural networks demonstrated their ability to provide remarkable performances on a wide range of supervised learning tasks (e.g., image classification) when trained on extensive collections of labeled data (e.g., ImageNet). However, creating such large datasets requires a considerable amount of resources, time, and effort. Such resources may not be available in many practical cases, limiting the adoption and the application of many deep learning methods. In a search for more data-efficient deep learning methods to overcome the need for large annotated datasets, there is a rising research interest in semi-supervised learning and its applications to deep neural networks to reduce the amount of labeled data required, by either developing novel methods or adopting existing semi-supervised learning frameworks for a deep learning setting. In this paper, we provide a comprehensive overview of deep semi-supervised learning, starting with an introduction to the field, followed by a summarization of the dominant semi-supervised approaches in deep learning.