OpenRaiser/Intern-Atlas huggingface dataset:
Overview of the IBM Neural Computer Architecture
Venue: arXiv cs.DC 2020
Year: 2020

The IBM Neural Computer (INC) is a highly flexible, re-configurable parallel processing system that is intended as a research and development platform for emerging machine intelligence algorithms and computational neuroscience. It consists of hundreds of programmable nodes, primarily based on Xilinx's Field Programmable Gate Array (FPGA) technology. The nodes are interconnected in a scalable 3d mesh topology. We overview INC, emphasizing unique features such as flexibility and scalability both in the types of computations performed and in the available modes of communication, enabling new machine intelligence approaches and learning strategies not well suited to the matrix manipulation/SIMD libraries that GPUs are optimized for. This paper describes the architecture of the machine and applications are to be described in detail elsewhere.
Value-Decomposition Networks based Distributed Interference Control in Multi-platoon Groupcast
Venue: arXiv cs.NI 2020
Year: 2020

Platooning is considered one of the most representative 5G use cases. Due to the small spacing within the platoon, the platoon needs more reliable transmission to guarantee driving safety while improving fuel and driving efficiency. However, efficient resource allocation between platoons has been a challenge, especially considering that the channel and power selected by each platoon will affect other platoons. Therefore, platoons need to coordinate with each other to ensure the groupcast quality of each platoon. To solve these challenges, we model the multi-platoon resource selection problem as Markov games and then propose a distributed resource allocation algorithm based on Value-Decomposition Networks. Our scheme utilizes the historical data of each platoon for centralized training. In distributed execution, agents only need their local observations to make decisions. At the same time, we decrease the training burden by sharing the neural network parameters. Simulation results show that the proposed algorithm has excellent convergence. Compared with another multi-agent algorithm (MARL) and random algorithm, our proposed solution can dramatically reduce the probability of platoon groupcast failure and improve the quality of platoon groupcast.
Volumization as a Natural Generalization of Weight Decay
Venue: arXiv cs.LG 2020
Year: 2020

We propose a novel regularization method, called \textit{volumization}, for neural networks. Inspired by physics, we define a physical volume for the weight parameters in neural networks, and we show that this method is an effective way of regularizing neural networks. Intuitively, this method interpolates between an $L_2$ and $L_\infty$ regularization. Therefore, weight decay and weight clipping become special cases of the proposed algorithm. We prove, on a toy example, that the essence of this method is a regularization technique to control bias-variance tradeoff. The method is shown to do well in the categories where the standard weight decay method is shown to work well, including improving the generalization of networks and preventing memorization. Moreover, we show that the volumization might lead to a simple method for training a neural network whose weight is binary or ternary.
Diseño e implementación de una meta-heurística multi-poblacional de optimización combinatoria enfocada a la resolución de problemas de asignación de rutas a vehículos
Venue: arXiv cs.AI 2020
Year: 2020

Transportation is an essential area in the nowadays society, both for business sector and citizenry. There are different kinds of transportation systems, each one with its own characteristics. In the same way, various areas of knowledge can deal efficiently with the transport planning. The majority of the problems related with the transport and logistics have common characteristics, so they can be modeled as optimization problems, being able to see them as special cases of other generic problems. These problems fit into the combinatorial optimization field. Much of the problems of this type have an exceptional complexity. A great amount of meta-heuristics can be found the literature, each one with its advantages and disadvantages. Due to the high complexity of combinatorial optimization problems, there is no technique able to solve all these problems optimally. This fact makes the fields of combinatorial optimization and vehicle routing problems be a hot topic of research. This doctoral thesis will focus its efforts on developing a new meta-heuristic to solve different kind of vehicle routing problems. The presented technique offers an added value compared to existing methods, either in relation to the performance, and the contribution of conceptual originality. With the aim of validating the proposed model, the results obtained by the developed meta-heuristic have been compared with the ones obtained by other four algorithms of similar philosophy. Four well-known routing problems have been used in this experimentation, as well as two classical combinatorial optimization problems. In addition to the comparisons based on parameters such as the mean, or the standard deviation, two different statistical tests have been carried out. Thanks to these tests it can be affirmed that the proposed meta-heuristic is competitive in terms of performance and conceptual originality.
Analog MIMO RoC Passive Relay for Indoor Deployments of Wireless Networks
Venue: arXiv cs.IT 2020
Year: 2020

Most of the indoor coverage issues arise from network deployments that are usually planned for outdoor scenarios. Moreover, the ever-growing number of devices with different Radio Access Technologies (RATs), expected for new 5G scenarios and to maintain compatibility with older cellular standards (mostly 3G/4G), worsen this situation thus calling for novel bandwidth-efficient, low-latency and cost-effective solutions for indoor coverage. To solve this problem, Centralized Radio Access Network (C-RAN) architectures have been proposed to provide dense and controlled coverage inside buildings. However, all-digital C-RAN solutions are complex and expensive when indoor layout constraints and device costs are considered. We discuss here an analog C-RAN architecture, referred to as Analog MIMO Radio-over-Copper (A-MIMO-RoC), that aims at distributing RF signals indoors over distances in the order of 50 m. The all-analog passive-only design presented here proves the feasibility of analog relaying of MIMO radio signals over LAN cables at frequency bandwidth values up to 400 MHz for multi-RAT applications. After asserting the feasibility of the A-MIMO-RoC platform, we present some experimental results obtained with the proposed architecture. These preliminary results show that the A-MIMO-RoC system is a valid solution towards the design of dedicated 4G/5G indoor wireless systems for the billions of buildings which nowadays still suffer from severe indoor coverage issues.
BlockMarkchain: A Secure Decentralized Data Market with a Constant Load on the Blockchain
Venue: arXiv cs.CR 2020
Year: 2020

In this paper, we develop BlockMarkchain, as a secure data market place, where individual data sellers can exchange certified data with buyers, in a secure environment, without any mutual trust among the parties, and without trusting on a third party, as a mediator. To develop this platform, we rely on a smart contract, deployed on a secure public blockchain. The main challenges here are to verify the validity of data and to prevent malicious behavior of the parties, while preserving the privacy of the data and taking into account the limited computing and storage resources available on the blockchain. In BlockMarkchain, the buyer has the option to dispute the honesty of the seller and prove the invalidity of the data to the smart contract. The smart contract evaluates the buyer's claim and punishes the dishonest party by forfeiting his/her deposit in favor of the honest party. BlockMarkchain enjoys several salient features including (i) the certified data has never been revealed on the public blockchain, (ii) the size of data posted on the blockchain, the load of computation on the blockchain, and the cost of communication with the blockchain is constant and negligible, and (iii) the computation cost of verifications on the parties is not expensive.
Privacy at Home: an Inquiry into Sensors and Robots for the Stay at Home Elderly
Venue: arXiv cs.RO 2020
Year: 2020

The elderly in the future will use smart house technology, sensors, and robots to stay at home longer. Privacy at home for these elderly is important. In this exploratory paper, we examine different understandings of privacy and use Palen and Dourish's framework to look at the negotiation of privacy along boundaries between a human at home, the robot, and its sensors. We select three dilemmas: turning sensors on and off, the robot seeing through walls, and machine learning. We discuss these dilemmas and also discuss ways the robot can help make the elderly more aware of privacy issues and to build trust.
Multi-faceted Trust-based Collaborative Filtering
Venue: arXiv cs.IR 2020
Year: 2020

Many collaborative recommender systems leverage social correlation theories to improve suggestion performance. However, they focus on explicit relations between users and they leave out other types of information that can contribute to determine users' global reputation; e.g., public recognition of reviewers' quality. We are interested in understanding if and when these additional types of feedback improve Top-N recommendation. For this purpose, we propose a multi-faceted trust model to integrate local trust, represented by social links, with various types of global trust evidence provided by social networks. We aim at identifying general classes of data in order to make our model applicable to different case studies. Then, we test the model by applying it to a variant of User-to-User Collaborative filtering (U2UCF) which supports the fusion of rating similarity, local trust derived from social relations, and multi-faceted reputation for rating prediction. We test our model on two datasets: the Yelp one publishes generic friend relations between users but provides different types of trust feedback, including user profile endorsements. The LibraryThing dataset offers fewer types of feedback but it provides more selective friend relations aimed at content sharing. The results of our experiments show that, on the Yelp dataset, our model outperforms both U2UCF and state-of-the-art trust-based recommenders that only use rating similarity and social relations. Differently, in the LibraryThing dataset, the combination of social relations and rating similarity achieves the best results. The lesson we learn is that multi-faceted trust can be a valuable type of information for recommendation. However, before using it in an application domain, an analysis of the type and amount of available trust evidence has to be done to assess its real impact on recommendation performance.
Derivation of Coupled PCA and SVD Learning Rules from a Newton Zero-Finding Framework
Venue: arXiv cs.NE 2020
Year: 2020

In coupled learning rules for PCA (principal component analysis) and SVD (singular value decomposition), the update of the estimates of eigenvectors or singular vectors is influenced by the estimates of eigenvalues or singular values, respectively. This coupled update mitigates the speed-stability problem since the update equations converge from all directions with approximately the same speed. A method to derive coupled learning rules from information criteria by Newton optimization is known. However, these information criteria have to be designed, offer no explanatory value, and can only impose Euclidean constraints on the vector estimates. Here we describe an alternative approach where coupled PCA and SVD learning rules can systematically be derived from a Newton zero-finding framework. The derivation starts from an objective function, combines the equations for its extrema with arbitrary constraints on the vector estimates, and solves the resulting vector zero-point equation using Newton's zero-finding method. To demonstrate the framework, we derive PCA and SVD learning rules with constant Euclidean length or constant sum of the vector estimates.
Scalable Variational Gaussian Process Regression Networks
Venue: arXiv cs.LG 2020
Year: 2020

Gaussian process regression networks (GPRN) are powerful Bayesian models for multi-output regression, but their inference is intractable. To address this issue, existing methods use a fully factorized structure (or a mixture of such structures) over all the outputs and latent functions for posterior approximation, which, however, can miss the strong posterior dependencies among the latent variables and hurt the inference quality. In addition, the updates of the variational parameters are inefficient and can be prohibitively expensive for a large number of outputs. To overcome these limitations, we propose a scalable variational inference algorithm for GPRN, which not only captures the abundant posterior dependencies but also is much more efficient for massive outputs. We tensorize the output space and introduce tensor/matrix-normal variational posteriors to capture the posterior correlations and to reduce the parameters. We jointly optimize all the parameters and exploit the inherent Kronecker product structure in the variational model evidence lower bound to accelerate the computation. We demonstrate the advantages of our method in several real-world applications.
FastPay: High-Performance Byzantine Fault Tolerant Settlement
Venue: arXiv cs.CR 2020
Year: 2020

FastPay allows a set of distributed authorities, some of which are Byzantine, to maintain a high-integrity and availability settlement system for pre-funded payments. It can be used to settle payments in a native unit of value (crypto-currency), or as a financial side-infrastructure to support retail payments in fiat currencies. FastPay is based on Byzantine Consistent Broadcast as its core primitive, foregoing the expenses of full atomic commit channels (consensus). The resulting system has low-latency for both confirmation and payment finality. Remarkably, each authority can be sharded across many machines to allow unbounded horizontal scalability. Our experiments demonstrate intra-continental confirmation latency of less than 100ms, making FastPay applicable to point of sale payments. In laboratory environments, we achieve over 80,000 transactions per second with 20 authorities---surpassing the requirements of current retail card payment networks, while significantly increasing their robustness.
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset, and quantify potential disparities using two approaches. First, we identify dangerous latent relationships that are captured by the contextual word embeddings using a fill-in-the-blank method with text from real clinical notes and a log probability bias score quantification. Second, we evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks that include detection of acute and chronic conditions. We find that classifiers trained from BERT representations exhibit statistically significant differences in performance, often favoring the majority group with regards to gender, language, ethnicity, and insurance status. Finally, we explore shortcomings of using adversarial debiasing to obfuscate subgroup information in contextual word embeddings, and recommend best practices for such deep embedding models in clinical settings.
Utilizing Deep Learning to Identify Drug Use on Twitter Data
Venue: arXiv cs.SI 2020
Year: 2020

The collection and examination of social media has become a useful mechanism for studying the mental activity and behavior tendencies of users. Through the analysis of collected Twitter data, models were developed for classifying drug-related tweets. Using topic pertaining keywords, such as slang and methods of drug consumption, a set of tweets was generated. Potential candidates were then preprocessed resulting in a dataset of 3,696,150 rows. The classification power of multiple methods was compared including support vector machines (SVM), XGBoost, and convolutional neural network (CNN) based classifiers. Rather than simple feature or attribute analysis, a deep learning approach was implemented to screen and analyze the tweets' semantic meaning. The two CNN-based classifiers presented the best result when compared against other methodologies. The first was trained with 2,661 manually labeled samples, while the other included synthetically generated tweets culminating in 12,142 samples. The accuracy scores were 76.35% and 82.31%, with an AUC of 0.90 and 0.91. Additionally, association rule mining showed that commonly mentioned drugs had a level of correspondence with frequently used illicit substances, proving the practical usefulness of the system. Lastly, the synthetically generated set provided increased scores, improving the classification capability and proving the worth of this methodology.
Cost-Sensitive BERT for Generalisable Sentence Classification with Imbalanced Data
Venue: arXiv cs.CL 2020
Year: 2020

The automatic identification of propaganda has gained significance in recent years due to technological and social changes in the way news is generated and consumed. That this task can be addressed effectively using BERT, a powerful new architecture which can be fine-tuned for text classification tasks, is not surprising. However, propaganda detection, like other tasks that deal with news documents and other forms of decontextualized social communication (e.g. sentiment analysis), inherently deals with data whose categories are simultaneously imbalanced and dissimilar. We show that BERT, while capable of handling imbalanced classes with no additional data augmentation, does not generalise well when the training and test data are sufficiently dissimilar (as is often the case with news sources, whose topics evolve over time). We show how to address this problem by providing a statistical measure of similarity between datasets and a method of incorporating cost-weighting into BERT when the training and test sets are dissimilar. We test these methods on the Propaganda Techniques Corpus (PTC) and achieve the second-highest score on sentence-level propaganda classification.
Improved Binary Artificial Bee Colony Algorithm
Venue: arXiv cs.NE 2020
Year: 2020

The Artificial Bee Colony (ABC) algorithm is an evolutionary optimization algorithm based on swarm intelligence and inspired by the honey bees' food search behavior. Since the ABC algorithm has been developed to achieve optimal solutions by searching in the continuous search space, modification is required to apply this method to binary optimization problems. In this paper, we improve the ABC algorithm to solve binary optimization problems and call it the improved binary Artificial Bee Colony (ibinABC). The proposed method consists of an update mechanism based on fitness values and processing different number of decision variables. Thus, we aim to prevent the ABC algorithm from getting stuck in a local minimum by increasing its exploration ability. We compare the ibinABC algorithm with three variants of the ABC and other meta-heuristic algorithms in the literature. For comparison, we use the wellknown OR-Library dataset containing 15 problem instances prepared for the uncapacitated facility location problem. Computational results show that the proposed method is superior to other methods in terms of convergence speed and robustness. The source code of the algorithm will be available on GitHub after reviewing process
Continuous Activity Maximization in Online Social Networks
Venue: arXiv cs.SI 2020
Year: 2020

Activity maximization is a task of seeking a small subset of users in a given social network that makes the expected total activity benefit maximized. This is a generalization of many real applications. In this paper, we extend activity maximization problem to that under the general marketing strategy $\vec{x}$, which is a $d$-dimensional vector from a lattice space and has probability $h_u(\vec{x})$ to activate a node $u$ as a seed. Based on that, we propose the continuous activity maximization (CAM) problem, where the domain is continuous and the seed set we select conforms to a certain probability distribution. It is a new topic to study the problem about information diffusion under the lattice constraint, thus, we address the problem systematically here. First, we analyze the hardness of CAM and how to compute the objective function of CAM accurately and effectively. We prove this objective function is monotone, but not DR-submodular and not DR-supermodular. Then, we develop a monotone and DR-submodular lower bound and upper bound of CAM, and apply sampling techniques to design three unbiased estimators for CAM, its lower bound and upper bound. Next, adapted from IMM algorithm and sandwich approximation framework, we obtain a data-dependent approximation ratio. This process can be considered as a general method to solve those maximization problem on lattice but not DR-submodular. Last, we conduct experiments on three real-world datasets to evaluate the correctness and effectiveness of our proposed algorithms.
Common-Knowledge Concept Recognition for SEVA
Venue: arXiv cs.CL 2020
Year: 2020

We build a common-knowledge concept recognition system for a Systems Engineer's Virtual Assistant (SEVA) which can be used for downstream tasks such as relation extraction, knowledge graph construction, and question-answering. The problem is formulated as a token classification task similar to named entity extraction. With the help of a domain expert and text processing methods, we construct a dataset annotated at the word-level by carefully defining a labelling scheme to train a sequence model to recognize systems engineering concepts. We use a pre-trained language model and fine-tune it with the labeled dataset of concepts. In addition, we also create some essential datasets for information such as abbreviations and definitions from the systems engineering domain. Finally, we construct a simple knowledge graph using these extracted concepts along with some hyponym relations.
Fastidious Attention Network for Navel Orange Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning achieves excellent performance in many domains, so we not only apply it to the navel orange semantic segmentation task to solve the two problems of distinguishing defect categories and identifying the stem end and blossom end, but also propose a fastidious attention mechanism to further improve model performance. This lightweight attention mechanism includes two learnable parameters, activations and thresholds, to capture long-range dependence. Specifically, the threshold picks out part of the spatial feature map and the activation excite this area. Based on activations and thresholds training from different types of feature maps, we design fastidious self-attention module (FSAM) and fastidious inter-attention module (FIAM). And then construct the Fastidious Attention Network (FANet), which uses U-Net as the backbone and embeds these two modules, to solve the problems with semantic segmentation for stem end, blossom end, flaw and ulcer. Compared with some state-of-the-art deep-learning-based networks under our navel orange dataset, experiments show that our network is the best performance with pixel accuracy 99.105%, mean accuracy 77.468%, mean IU 70.375% and frequency weighted IU 98.335%. And embedded modules show better discrimination of 5 categories including background, especially the IU of flaw is increased by 3.165%.