OpenRaiser/Intern-Atlas huggingface dataset:
Benchmarking for Metaheuristic Black-Box Optimization: Perspectives and Open Challenges
Venue: arXiv cs.NE 2020
Year: 2020

Research on new optimization algorithms is often funded based on the motivation that such algorithms might improve the capabilities to deal with real-world and industrially relevant optimization challenges. Besides a huge variety of different evolutionary and metaheuristic optimization algorithms, also a large number of test problems and benchmark suites have been developed and used for comparative assessments of algorithms, in the context of global, continuous, and black-box optimization. For many of the commonly used synthetic benchmark problems or artificial fitness landscapes, there are however, no methods available, to relate the resulting algorithm performance assessments to technologically relevant real-world optimization problems, or vice versa. Also, from a theoretical perspective, many of the commonly used benchmark problems and approaches have little to no generalization value. Based on a mini-review of publications with critical comments, advice, and new approaches, this communication aims to give a constructive perspective on several open challenges and prospective research directions related to systematic and generalizable benchmarking for black-box optimization.
Index Coding in Vehicle to Vehicle Communication
Venue: arXiv cs.IT 2020
Year: 2020

Vehicle to Vehicle (V2V) communication phase is an integral part of collaborative message dissemination in vehicular ad-hoc networks (VANETs). In this work, we apply index coding techniques to reduce the number of transmissions required for data exchange. The index coding problem has a sender, which tries to meet the demands of several receivers in a minimum number of transmissions. All these receivers have some prior knowledge of the messages, known as the side-information. In this work, we consider a particular case of the index coding problem, where multiple nodes want to share information among them. Under this set up, lower bound on the number of transmissions is established when the cardinality of side-information is the same. An optimal solution to achieve the bound in a special case of VANET scenario is presented. For this special case, we consider the link between the nodes to be error-prone, and in this setting, we construct optimal linear error correcting index codes.
Reasoning with Contextual Knowledge and Influence Diagrams
Venue: arXiv cs.LO 2020
Year: 2020

Influence diagrams (IDs) are well-known formalisms extending Bayesian networks to model decision situations under uncertainty. Although they are convenient as a decision theoretic tool, their knowledge representation ability is limited in capturing other crucial notions such as logical consistency. We complement IDs with the light-weight description logic (DL) EL to overcome such limitations. We consider a setup where DL axioms hold in some contexts, yet the actual context is uncertain. The framework benefits from the convenience of using DL as a domain knowledge representation language and the modelling strength of IDs to deal with decisions over contexts in the presence of contextual uncertainty. We define related reasoning problems and study their computational complexity.
Lightweight Temporal Self-Attention for Classifying Satellite Image Time Series
Venue: arXiv cs.CV 2020
Year: 2020

The increasing accessibility and precision of Earth observation satellite data offers considerable opportunities for industrial and state actors alike. This calls however for efficient methods able to process time-series on a global scale. Building on recent work employing multi-headed self-attention mechanisms to classify remote sensing time sequences, we propose a modification of the Temporal Attention Encoder. In our network, the channels of the temporal inputs are distributed among several compact attention heads operating in parallel. Each head extracts highly-specialized temporal features which are in turn concatenated into a single representation. Our approach outperforms other state-of-the-art time series classification algorithms on an open-access satellite image dataset, while using significantly fewer parameters and with a reduced computational complexity.
HydroNets: Leveraging River Structure for Hydrologic Modeling
Venue: arXiv cs.LG 2020
Year: 2020

Accurate and scalable hydrologic models are essential building blocks of several important applications, from water resource management to timely flood warnings. However, as the climate changes, precipitation and rainfall-runoff pattern variations become more extreme, and accurate training data that can account for the resulting distributional shifts become more scarce. In this work we present a novel family of hydrologic models, called HydroNets, which leverages river network structure. HydroNets are deep neural network models designed to exploit both basin specific rainfall-runoff signals, and upstream network dynamics, which can lead to improved predictions at longer horizons. The injection of the river structure prior knowledge reduces sample complexity and allows for scalable and more accurate hydrologic modeling even with only a few years of data. We present an empirical study over two large basins in India that convincingly support the proposed model and its advantages.
Effects for Efficiency: Asymptotic Speedup with First-Class Control
Venue: arXiv cs.PL 2020
Year: 2020

We study the fundamental efficiency of delimited control. Specifically, we show that effect handlers enable an asymptotic improvement in runtime complexity for a certain class of functions. We consider the generic count problem using a pure PCF-like base language $λ_b$ and its extension with effect handlers $λ_h$. We show that $λ_h$ admits an asymptotically more efficient implementation of generic count than any $λ_b$ implementation. We also show that this efficiency gap remains when $λ_b$ is extended with mutable state. To our knowledge this result is the first of its kind for control operators.
On the Distributed Construction of Stable Networks in Polylogarithmic Parallel Time
Venue: arXiv cs.DC 2020
Year: 2020

We study the class of networks which can be created in polylogarithmic parallel time by network constructors: groups of anonymous agents that interact randomly under a uniform random scheduler with the ability to form connections between each other. Starting from an empty network, the goal is to construct a stable network which belongs to a given family. We prove that the class of trees where each node has any k >= 2 children can be constructed in O(log n) parallel time with high probability. We show that constructing networks which are k-regular is Omega(n) time, but a minimal relaxation to (l, k)-regular networks, where l = k - 1 can be constructed in polylogarithmic parallel time for any fixed k, where k > 2. We further demonstrate that when the finite-state assumption is relaxed and k is allowed to grow with n, then k = log log n acts as a threshold above which network construction is again polynomial time. We use this to provide a partial characterisation of the class of polylogarithmic time network constructors.
Group Ensemble: Learning an Ensemble of ConvNets in a single ConvNet
Venue: arXiv cs.CV 2020
Year: 2020

Ensemble learning is a general technique to improve accuracy in machine learning. However, the heavy computation of a ConvNets ensemble limits its usage in deep learning. In this paper, we present Group Ensemble Network (GENet), an architecture incorporating an ensemble of ConvNets in a single ConvNet. Through a shared-base and multi-head structure, GENet is divided into several groups to make explicit ensemble learning possible in a single ConvNet. Owing to group convolution and the shared-base, GENet can fully leverage the advantage of explicit ensemble learning while retaining the same computation as a single ConvNet. Additionally, we present Group Averaging, Group Wagging and Group Boosting as three different strategies to aggregate these ensemble members. Finally, GENet outperforms larger single networks, standard ensembles of smaller networks, and other recent state-of-the-art methods on CIFAR and ImageNet. Specifically, group ensemble reduces the top-1 error by 1.83% for ResNeXt-50 on ImageNet. We also demonstrate its effectiveness on action recognition and object detection tasks.
Build2Vec: Building Representation in Vector Space
Venue: arXiv cs.CY 2020
Year: 2020

In this paper, we represent a methodology of a graph embeddings algorithm that is used to transform labeled property graphs obtained from a Building Information Model (BIM). Industrial Foundation Classes (IFC) is a standard schema for BIM, which is utilized to convert the building data into a graph representation. We used node2Vec with biased random walks to extract semantic similarities between different building components and represent them in a multi-dimensional vector space. A case study implementation is conducted on a net-zero-energy building located at the National University of Singapore (SDE4). This approach shows promising machine learning applications in capturing the semantic relations and similarities of different building objects, more specifically, spatial and spatio-temporal data.
Bandit Linear Control
Venue: arXiv cs.LG 2020
Year: 2020

We consider the problem of controlling a known linear dynamical system under stochastic noise, adversarially chosen costs, and bandit feedback. Unlike the full feedback setting where the entire cost function is revealed after each decision, here only the cost incurred by the learner is observed. We present a new and efficient algorithm that, for strongly convex and smooth costs, obtains regret that grows with the square root of the time horizon $T$. We also give extensions of this result to general convex, possibly non-smooth costs, and to non-stochastic system noise. A key component of our algorithm is a new technique for addressing bandit optimization of loss functions with memory.
A simple bipartite graph projection model for clustering in networks
Venue: arXiv cs.SI 2020
Year: 2020

Graph datasets are frequently constructed by a projection of a bipartite graph, where two nodes are connected in the projection if they share a common neighbor in the bipartite graph; for example, a coauthorship graph is a projection of an author-publication bipartite graph. Analyzing the structure of the projected graph is common, but we do not have a good understanding of the consequences of the projection on such analyses. Here, we propose and analyze a random graph model to study what properties we can expect from the projection step. Our model is based on a Chung-Lu random graph for constructing the bipartite representation, which enables us to rigorously analyze the projected graph. We show that common network properties such as sparsity, heavy-tailed degree distributions, local clustering at nodes, the inverse relationship between node degree, and global transitivity can be explained and analyzed through this simple model. We also develop a fast sampling algorithm for our model, which we show is provably optimal for certain input distributions. Numerical simulations where model parameters come from real-world datasets show that much of the clustering behavior in some datasets can just be explained by the projection step.
TeleVital: Enhancing the quality of contactless health assessment
Venue: arXiv cs.HC 2020
Year: 2020

In the midst of rising positive cases of COVID-19, the hospitals face a newfound difficulty to prioritize on their patients and accommodate them. Moreover, crowding of patients at hospitals pose a threat to the healthcare workers and other patients at the hospital. With that in mind, a non-contact method of measuring the necessary vitals such as heart rate, respiratory rate and SPO$_2$ will prove highly beneficial for the hospitals to tackle this issue. This paper discusses our approach in achieving the non-contact measurement of vitals with the sole help of a webcam and further our design of an e-hospital platform for doctors and patients to attend appointments virtually. The platform also provides the doctor with an option to provide with voice-based prescriptions or digital prescriptions, to simplify the daily, exhausting routine of a doctor.
Age-Oriented Face Synthesis with Conditional Discriminator Pool and Adversarial Triplet Loss
Venue: arXiv cs.CV 2020
Year: 2020

The vanilla Generative Adversarial Networks (GAN) are commonly used to generate realistic images depicting aged and rejuvenated faces. However, the performance of such vanilla GANs in the age-oriented face synthesis task is often compromised by the mode collapse issue, which may result in the generation of faces with minimal variations and a poor synthesis accuracy. In addition, recent age-oriented face synthesis methods use the L1 or L2 constraint to preserve the identity information on synthesized faces, which implicitly limits the identity permanence capabilities when these constraints are associated with a trivial weighting factor. In this paper, we propose a method for the age-oriented face synthesis task that achieves a high synthesis accuracy with strong identity permanence capabilities. Specifically, to achieve a high synthesis accuracy, our method tackles the mode collapse issue with a novel Conditional Discriminator Pool (CDP), which consists of multiple discriminators, each targeting one particular age category. To achieve strong identity permanence capabilities, our method uses a novel Adversarial Triplet loss. This loss, which is based on the Triplet loss, adds a ranking operation to further pull the positive embedding towards the anchor embedding resulting in significantly reduced intra-class variances in the feature space. Through extensive experiments, we show that our proposed method outperforms state-of-the-art methods in terms of synthesis accuracy and identity permanence capabilities, qualitatively and quantitatively.
A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Deep neural networks are typically trained under a supervised learning framework where a model learns a single task using labeled data. Instead of relying solely on labeled data, practitioners can harness unlabeled or related data to improve model performance, which is often more accessible and ubiquitous. Self-supervised pre-training for transfer learning is becoming an increasingly popular technique to improve state-of-the-art results using unlabeled data. It involves first pre-training a model on a large amount of unlabeled data, then adapting the model to target tasks of interest. In this review, we survey self-supervised learning methods and their applications within the sequential transfer learning framework. We provide an overview of the taxonomy for self-supervised learning and transfer learning, and highlight some prominent methods for designing pre-training tasks across different domains. Finally, we discuss recent trends and suggest areas for future investigation.
Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge
Venue: arXiv cs.CL 2020
Year: 2020

Massive language models are the core of modern NLP modeling and have been shown to encode impressive amounts of commonsense and factual information. However, that knowledge exists only within the latent parameters of the model, inaccessible to inspection and interpretation, and even worse, factual information memorized from the training corpora is likely to become stale as the world changes. Knowledge stored as parameters will also inevitably exhibit all of the biases inherent in the source materials. To address these problems, we develop a neural language model that includes an explicit interface between symbolically interpretable factual information and subsymbolic neural knowledge. We show that this model dramatically improves performance on two knowledge-intensive question-answering tasks. More interestingly, the model can be updated without re-training by manipulating its symbolic representations. In particular this model allows us to add new facts and overwrite existing ones in ways that are not possible for earlier models.
Reconfigurable Intelligent Surface Assisted Device-to-Device Communications
Venue: arXiv cs.IT 2020
Year: 2020

With the evolution of the 5G, 6G and beyond, device-to-device (D2D) communication has been developed as an energy-, and spectrum-efficient solution. In cellular network, D2D links need to share the same spectrum resources with the cellular link. A reconfigurable intelligent surface (RIS) can reconfigure the phase shifts of elements and create favorable beam steering, which can mitigate aggravated interference caused by D2D links. In this paper, we study a RIS-assisted single cell uplink communication network scenario, where the cellular link and multiple D2D links utilize direct propagation and reflecting one-hop propagation. The problem of maximizing the total system rate is formulated by jointly optimizing transmission powers of all links and discrete phase shifts of all elements. The formulated problem is an NP-hard mixed integer non-convex non-linear problem. To obtain practical solutions, we capitalize on alternating maximization and the problem is decomposed into two sub-problems. For the power allocation, the problem is a difference of concave functions (DC) problem, which is solved with the gradient descent method. For the phase shift, a local search algorithm with lower complexity is utilized. Simulation results show that deploying RIS and optimizing the phase shifts have a significant effect on mitigating D2D network interference.
Design and Analysis of 6-DOF Triple Scissor Extender Robots with Applications in Aircraft Assembly
Venue: arXiv cs.RO 2020
Year: 2020

A new type of parallel robot mechanism with an extendable structure is presented, and its kinematic properties and design parameters are analyzed. The Triple Scissor Extender (TSE) is a 6 Degree-Of-Freedom robotic mechanism for reaching high ceilings and positioning an end effector. Three scissor mechanisms are arranged in parallel, with the bottom ends coupled to linear slides, and the top vertex attached to an end effector plate. Arbitrary positions and orientations of the end effector can be achieved through the coordinated motion of the six linear actuators located at the base. By changing key geometric parameters, the TSE's design can yield a specific desired workspace volume and differential motion behavior. A general kinematic model for diverse TSEs is derived, and the kinematic properties, including workspace, singularity, and the Jacobian singular values, are evaluated. From these expressions, four key design parameters are identified, and their sensitivity upon the workspace volume and the Jacobian singular values is analyzed. A case study in autonomous aircraft assembly is presented using the insights gained from the design parameter studies.
Deep brain state classification of MEG data
Venue: arXiv cs.LG 2020
Year: 2020

Neuroimaging techniques have shown to be useful when studying the brain's activity. This paper uses Magnetoencephalography (MEG) data, provided by the Human Connectome Project (HCP), in combination with various deep artificial neural network models to perform brain decoding. More specifically, here we investigate to which extent can we infer the task performed by a subject based on its MEG data. Three models based on compact convolution, combined convolutional and long short-term architecture as well as a model based on multi-view learning that aims at fusing the outputs of the two stream networks are proposed and examined. These models exploit the spatio-temporal MEG data for learning new representations that are used to decode the relevant tasks across subjects. In order to realize the most relevant features of the input signals, two attention mechanisms, i.e. self and global attention, are incorporated in all the models. The experimental results of cross subject multi-class classification on the studied MEG dataset show that the inclusion of attention improves the generalization of the models across subjects.
ER model Partitioning: Towards Trustworthy Automated Systems Development
Venue: arXiv cs.DB 2020
Year: 2020

In database development, a conceptual model is created, in the form of an Entity-relationship(ER) model, and transformed to a relational database schema (RDS) to create the database. However, some important information represented on the ER model may not be transformed and represented on the RDS. This situation causes a loss of information during the transformation process. With a view to preserving information, in our previous study, we standardized the transformation process as a one-to-one and onto mapping from the ER model to the RDS. For this purpose, we modified the ER model and the transformation algorithm resolving some deficiencies existed in them. Since the mapping was established using a few real-world cases as a basis and for verification purposes, a formal-proof is necessary to validate the work. Thus, the ongoing research aiming to create a proof will show how a given ER model can be partitioned into a unique set of segments and use it to represent the ER model itself. How the findings can be used to complete the proof in the future will also be explained. Significance of the research on automating database development, teaching conceptual modeling, and using formal methods will also be discussed.
A Novel DNN Training Framework via Data Sampling and Multi-Task Optimization
Venue: arXiv cs.NE 2020
Year: 2020

Conventional DNN training paradigms typically rely on one training set and one validation set, obtained by partitioning an annotated dataset used for training, namely gross training set, in a certain way. The training set is used for training the model while the validation set is used to estimate the generalization performance of the trained model as the training proceeds to avoid over-fitting. There exist two major issues in this paradigm. Firstly, the validation set may hardly guarantee an unbiased estimate of generalization performance due to potential mismatching with test data. Secondly, training a DNN corresponds to solve a complex optimization problem, which is prone to getting trapped into inferior local optima and thus leads to undesired training results. To address these issues, we propose a novel DNN training framework. It generates multiple pairs of training and validation sets from the gross training set via random splitting, trains a DNN model of a pre-specified structure on each pair while making the useful knowledge (e.g., promising network parameters) obtained from one model training process to be transferred to other model training processes via multi-task optimization, and outputs the best, among all trained models, which has the overall best performance across the validation sets from all pairs. The knowledge transfer mechanism featured in this new framework can not only enhance training effectiveness by helping the model training process to escape from local optima but also improve on generalization performance via implicit regularization imposed on one model training process from other model training processes. We implement the proposed framework, parallelize the implementation on a GPU cluster, and apply it to train several widely used DNN models. Experimental results demonstrate the superiority of the proposed framework over the conventional training paradigm.