OpenRaiser/Intern-Atlas huggingface dataset:
Match4Rec: A Novel Recommendation Algorithm Based on Bidirectional Encoder Representation with the Matching Task
Venue: arXiv cs.SI 2020
Year: 2020

Characterizing users' interests accurately plays a significant role in an effective recommender system. The sequential recommender system can learn powerful hidden representations of users from successive user-item interactions and dynamic users' preferences. To analyze such sequential data, conventional methods mainly include Markov Chains (MCs) and Recurrent Neural Networks (RNNs). Recently, the use of self-attention mechanisms and bi-directional architectures have gained much attention. However, there still exists a major limitation in previous works that they only model the user's main purposes in the behavioral sequences separately and locally, and they lack the global representation of the user's whole sequential behavior. To address this limitation, we propose a novel bidirectional sequential recommendation algorithm that integrates the user's local purposes with the global preference by additive supervision of the matching task. We combine the mask task with the matching task in the training process of the bidirectional encoder. A new sample production method is also introduced to alleviate the effect of mask noise. Our proposed model can not only learn bidirectional semantics from users' behavioral sequences but also explicitly produces user representations to capture user's global preference. Extensive empirical studies demonstrate our approach considerably outperforms various state-of-the-art models.
Discovering Bilingual Lexicons in Polyglot Word Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

Bilingual lexicons and phrase tables are critical resources for modern Machine Translation systems. Although recent results show that without any seed lexicon or parallel data, highly accurate bilingual lexicons can be learned using unsupervised methods, such methods rely on the existence of large, clean monolingual corpora. In this work, we utilize a single Skip-gram model trained on a multilingual corpus yielding polyglot word embeddings, and present a novel finding that a surprisingly simple constrained nearest-neighbor sampling technique in this embedding space can retrieve bilingual lexicons, even in harsh social media data sets predominantly written in English and Romanized Hindi and often exhibiting code switching. Our method does not require monolingual corpora, seed lexicons, or any other such resources. Additionally, across three European language pairs, we observe that polyglot word embeddings indeed learn a rich semantic representation of words and substantial bilingual lexicons can be retrieved using our constrained nearest neighbor sampling. We investigate potential reasons and downstream applications in settings spanning both clean texts and noisy social media data sets, and in both resource-rich and under-resourced language pairs.
Active Local Learning
Venue: arXiv cs.LG 2020
Year: 2020

In this work we consider active local learning: given a query point $x$, and active access to an unlabeled training set $S$, output the prediction $h(x)$ of a near-optimal $h \in H$ using significantly fewer labels than would be needed to actually learn $h$ fully. In particular, the number of label queries should be independent of the complexity of $H$, and the function $h$ should be well-defined, independent of $x$. This immediately also implies an algorithm for distance estimation: estimating the value $opt(H)$ from many fewer labels than needed to actually learn a near-optimal $h \in H$, by running local learning on a few random query points and computing the average error. For the hypothesis class consisting of functions supported on the interval $[0,1]$ with Lipschitz constant bounded by $L$, we present an algorithm that makes $O(({1 / ε^6}) \log(1/ε))$ label queries from an unlabeled pool of $O(({L / ε^4})\log(1/ε))$ samples. It estimates the distance to the best hypothesis in the class to an additive error of $ε$ for an arbitrary underlying distribution. We further generalize our algorithm to more than one dimensions. We emphasize that the number of labels used is independent of the complexity of the hypothesis class which depends on $L$. Furthermore, we give an algorithm to locally estimate the values of a near-optimal function at a few query points of interest with number of labels independent of $L$. We also consider the related problem of approximating the minimum error that can be achieved by the Nadaraya-Watson estimator under a linear diagonal transformation with eigenvalues coming from a small range. For a $d$-dimensional pointset of size $N$, our algorithm achieves an additive approximation of $ε$, makes $\tilde{O}({d}/{ε^2})$ queries and runs in $\tilde{O}({d^2}/{ε^{d+4}}+{dN}/{ε^2})$ time.
Constraint-Logic Object-Oriented Programming with Free Arrays
Venue: arXiv cs.PL 2020
Year: 2020

Constraint-logic object-oriented programming provides a useful symbiosis between object-oriented programming and constraint-logic search. The ability to use logic variables, constraints, non-deterministic search, and object-oriented programming in an integrated way facilitates the combination of search-related program parts and other business logic in object-oriented applications. With this work we conceptualize array-typed logic variables ("free arrays"), thus completing the set of types that logic variables can assume in constraint-logic object-oriented programming. Free arrays exhibit interesting properties, such as indeterminate lengths and non-deterministic accesses to array elements.
A Cost-Effective Person-Following System for Assistive Unmanned Vehicles with Deep Learning at the Edge
Venue: arXiv cs.RO 2020
Year: 2020

The vital statistics of the last century highlight a sharp increment of the average age of the world population with a consequent growth of the number of older people. Service robotics applications have the potentiality to provide systems and tools to support the autonomous and self-sufficient older adults in their houses in everyday life, thereby avoiding the task of monitoring them with third parties. In this context, we propose a cost-effective modular solution to detect and follow a person in an indoor, domestic environment. We exploited the latest advancements in deep learning optimization techniques, and we compared different neural network accelerators to provide a robust and flexible person-following system at the edge. Our proposed cost-effective and power-efficient solution is fully-integrable with pre-existing navigation stacks and creates the foundations for the development of fully-autonomous and self-contained service robotics applications.
Active exploration for body model learning through self-touch on a humanoid robot with artificial skin
Venue: arXiv cs.RO 2020
Year: 2020

The mechanisms of infant development are far from understood. Learning about one's own body is likely a foundation for subsequent development. Here we look specifically at the problem of how spontaneous touches to the body in early infancy may give rise to first body models and bootstrap further development such as reaching competence. Unlike visually elicited reaching, reaching to own body requires connections of the tactile and motor space only, bypassing vision. Still, the problems of high dimensionality and redundancy of the motor system persist. In this work, we present an embodied computational model on a simulated humanoid robot with artificial sensitive skin on large areas of its body. The robot should autonomously develop the capacity to reach for every tactile sensor on its body. To do this efficiently, we employ the computational framework of intrinsic motivations and variants of goal babbling, as opposed to motor babbling, that prove to make the exploration process faster and alleviate the ill-posedness of learning inverse kinematics. Based on our results, we discuss the next steps in relation to infant studies: what information will be necessary to further ground this computational model in behavioral data.
ROS-Neuro Integration of Deep Convolutional Autoencoders for EEG Signal Compression in Real-time BCIs
Venue: arXiv cs.LG 2020
Year: 2020

Typical EEG-based BCI applications require the computation of complex functions over the noisy EEG channels to be carried out in an efficient way. Deep learning algorithms are capable of learning flexible nonlinear functions directly from data, and their constant processing latency is perfect for their deployment into online BCI systems. However, it is crucial for the jitter of the processing system to be as low as possible, in order to avoid unpredictable behaviour that can ruin the system's overall usability. In this paper, we present a novel encoding method, based on on deep convolutional autoencoders, that is able to perform efficient compression of the raw EEG inputs. We deploy our model in a ROS-Neuro node, thus making it suitable for the integration in ROS-based BCI and robotic systems in real world scenarios. The experimental results show that our system is capable to generate meaningful compressed encoding preserving to original information contained in the raw input. They also show that the ROS-Neuro node is able to produce such encodings at a steady rate, with minimal jitter. We believe that our system can represent an important step towards the development of an effective BCI processing pipeline fully standardized in ROS-Neuro framework.
Direct Volume Rendering with Nonparametric Models of Uncertainty
Venue: arXiv cs.GR 2020
Year: 2020

We present a nonparametric statistical framework for the quantification, analysis, and propagation of data uncertainty in direct volume rendering (DVR). The state-of-the-art statistical DVR framework allows for preserving the transfer function (TF) of the ground truth function when visualizing uncertain data; however, the existing framework is restricted to parametric models of uncertainty. In this paper, we address the limitations of the existing DVR framework by extending the DVR framework for nonparametric distributions. We exploit the quantile interpolation technique to derive probability distributions representing uncertainty in viewing-ray sample intensities in closed form, which allows for accurate and efficient computation. We evaluate our proposed nonparametric statistical models through qualitative and quantitative comparisons with the mean-field and parametric statistical models, such as uniform and Gaussian, as well as Gaussian mixtures. In addition, we present an extension of the state-of-the-art rendering parametric framework to 2D TFs for improved DVR classifications. We show the applicability of our uncertainty quantification framework to ensemble, downsampled, and bivariate versions of scalar field datasets.
Change-Point Analysis of Cyberbullying-Related Twitter Discussions During COVID-19
Venue: arXiv cs.SI 2020
Year: 2020

Due to the outbreak of COVID-19, users are increasingly turning to online services. An increase in social media usage has also been observed, leading to the suspicion that this has also raised cyberbullying. In this initial work, we explore the possibility of an increase in cyberbullying incidents due to the pandemic and high social media usage. To evaluate this trend, we collected 454,046 cyberbullying-related public tweets posted between January 1st, 2020 -- June 7th, 2020. We summarize the tweets containing multiple keywords into their daily counts. Our analysis showed the existence of at most one statistically significant changepoint for most of these keywords, which were primarily located around the end of March. Almost all these changepoint time-locations can be attributed to COVID-19, which substantiates our initial hypothesis of an increase in cyberbullying through analysis of discussions over Twitter.
Connecting Web Event Forecasting with Anomaly Detection: A Case Study on Enterprise Web Applications Using Self-Supervised Neural Networks
Venue: arXiv cs.CR 2020
Year: 2020

Recently web applications have been widely used in enterprises to assist employees in providing effective and efficient business processes. Forecasting upcoming web events in enterprise web applications can be beneficial in many ways, such as efficient caching and recommendation. In this paper, we present a web event forecasting approach, DeepEvent, in enterprise web applications for better anomaly detection. DeepEvent includes three key features: web-specific neural networks to take into account the characteristics of sequential web events, self-supervised learning techniques to overcome the scarcity of labeled data, and sequence embedding techniques to integrate contextual events and capture dependencies among web events. We evaluate DeepEvent on web events collected from six real-world enterprise web applications. Our experimental results demonstrate that DeepEvent is effective in forecasting sequential web events and detecting web based anomalies. DeepEvent provides a context-based system for researchers and practitioners to better forecast web events with situational awareness.
Langevin Cooling for Domain Translation
Venue: arXiv cs.LG 2020
Year: 2020

Domain translation is the task of finding correspondence between two domains. Several Deep Neural Network (DNN) models, e.g., CycleGAN and cross-lingual language models, have shown remarkable successes on this task under the unsupervised setting---the mappings between the domains are learned from two independent sets of training data in both domains (without paired samples). However, those methods typically do not perform well on a significant proportion of test samples. In this paper, we hypothesize that many of such unsuccessful samples lie at the fringe---relatively low-density areas---of data distribution, where the DNN was not trained very well, and propose to perform Langevin dynamics to bring such fringe samples towards high density areas. We demonstrate qualitatively and quantitatively that our strategy, called Langevin Cooling (L-Cool), enhances state-of-the-art methods in image translation and language translation tasks.
Under Water Waste Cleaning by Mobile Edge Computing and Intelligent Image Processing Based Robotic Fish
Venue: arXiv cs.NI 2020
Year: 2020

As water pollution is a serious threat to underwater resources, i.e., underwater plants and species, we focus on protecting the resources by cleaning the non-biodegradable waste from the water. The waste can be recycled for further usage. Here we design a robotic fish which mainly comprises optical biosensor, camera module, piston module, and wireless transceiver. By exploiting the LTE and 5G network architecture, the fish stores the information about the underwater waste in the nearest mobile edge computing server as well as in the centralized cloud server. Finally, when the fish clears the underwater waste, it offloads the captured image of the located object to the mobile edge computing server or sometimes to the cloud server for making a decision. The servers employ intelligent image processing technology and an adaptive learning process to make a decision. However, if the servers fail to make a decision, then the fish utilizes its optical biosensor. By this scheme, the time delay for clearing any water body is minimized and the waste collection capacity of the fish is maximized. This technique can effectively help the government or municipal personnel for making clean water without manual efforts.
ArchiveSafe: Mass-Leakage-Resistant Storage from Proof-of-Work
Venue: arXiv cs.CR 2020
Year: 2020

Data breaches-mass leakage of stored information-are a major security concern. Encryption can provide confidentiality, but encryption depends on a key which, if compromised, allows the attacker to decrypt everything, effectively instantly. Security of encrypted data thus becomes a question of protecting the encryption keys. In this paper, we propose using keyless encryption to construct a mass leakage resistant archiving system, where decryption of a file is only possible after the requester, whether an authorized user or an adversary, completes a proof of work in the form of solving a cryptographic puzzle. This proposal is geared towards protection of infrequently-accessed archival data, where any one file may not require too much work to decrypt, decryption of a large number of files-mass leakage-becomes increasingly expensive for an attacker. We present a prototype implementation realized as a user-space file system driver for Linux. We report experimental results of system behaviour under different file sizes and puzzle difficulty levels. Our keyless encryption technique can be added as a layer on top of traditional encryption: together they provide strong security against adversaries without the key and resistance against mass decryption by an attacker.
Adversarial Eigen Attack on Black-Box Models
Venue: arXiv cs.LG 2020
Year: 2020

Black-box adversarial attack has attracted a lot of research interests for its practical use in AI safety. Compared with the white-box attack, a black-box setting is more difficult for less available information related to the attacked model and the additional constraint on the query budget. A general way to improve the attack efficiency is to draw support from a pre-trained transferable white-box model. In this paper, we propose a novel setting of transferable black-box attack: attackers may use external information from a pre-trained model with available network parameters, however, different from previous studies, no additional training data is permitted to further change or tune the pre-trained model. To this end, we further propose a new algorithm, EigenBA to tackle this problem. Our method aims to explore more gradient information of the black-box model, and promote the attack efficiency, while keeping the perturbation to the original attacked image small, by leveraging the Jacobian matrix of the pre-trained white-box model. We show the optimal perturbations are closely related to the right singular vectors of the Jacobian matrix. Further experiments on ImageNet and CIFAR-10 show that even the unlearnable pre-trained white-box model could also significantly boost the efficiency of the black-box attack and our proposed method could further improve the attack efficiency.
A Framework For Contrastive Self-Supervised Learning And Designing A New Approach
Venue: arXiv cs.CV 2020
Year: 2020

Contrastive self-supervised learning (CSL) is an approach to learn useful representations by solving a pretext task that selects and compares anchor, negative and positive (APN) features from an unlabeled dataset. We present a conceptual framework that characterizes CSL approaches in five aspects (1) data augmentation pipeline, (2) encoder selection, (3) representation extraction, (4) similarity measure, and (5) loss function. We analyze three leading CSL approaches--AMDIM, CPC, and SimCLR--, and show that despite different motivations, they are special cases under this framework. We show the utility of our framework by designing Yet Another DIM (YADIM) which achieves competitive results on CIFAR-10, STL-10 and ImageNet, and is more robust to the choice of encoder and the representation extraction strategy. To support ongoing CSL research, we release the PyTorch implementation of this conceptual framework along with standardized implementations of AMDIM, CPC (V2), SimCLR, BYOL, Moco (V2) and YADIM.
A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
Venue: arXiv cs.CV 2020
Year: 2020

Large-scale labeled training datasets have enabled deep neural networks to excel across a wide range of benchmark vision tasks. However, in many applications, it is prohibitively expensive and time-consuming to obtain large quantities of labeled data. To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain. Unfortunately, direct transfer across domains often performs poorly due to the presence of domain shift or dataset bias. Domain adaptation is a machine learning paradigm that aims to learn a model from a source domain that can perform well on a different (but related) target domain. In this paper, we review the latest single-source deep unsupervised domain adaptation methods focused on visual tasks and discuss new perspectives for future research. We begin with the definitions of different domain adaptation strategies and the descriptions of existing benchmark datasets. We then summarize and compare different categories of single-source unsupervised domain adaptation methods, including discrepancy-based methods, adversarial discriminative methods, adversarial generative methods, and self-supervision-based methods. Finally, we discuss future research directions with challenges and possible solutions.
Reinforcement Learning-based Black-Box Evasion Attacks to Link Prediction in Dynamic Graphs
Venue: arXiv cs.CR 2020
Year: 2020

Link prediction in dynamic graphs (LPDG) is an important research problem that has diverse applications such as online recommendations, studies on disease contagion, organizational studies, etc. Various LPDG methods based on graph embedding and graph neural networks have been recently proposed and achieved state-of-the-art performance. In this paper, we study the vulnerability of LPDG methods and propose the first practical black-box evasion attack. Specifically, given a trained LPDG model, our attack aims to perturb the graph structure, without knowing to model parameters, model architecture, etc., such that the LPDG model makes as many wrong predicted links as possible. We design our attack based on a stochastic policy-based RL algorithm. Moreover, we evaluate our attack on three real-world graph datasets from different application domains. Experimental results show that our attack is both effective and efficient.
LoRaWAN Temperature Sensors for Local Government Asset Management
Venue: arXiv cs.CY 2020
Year: 2020

The purpose of this project is to investigate the suitability of using LoRaWAN technology to conduct temperature studies on local government assets in Australian metropolitan and residential areas. Temperature sensing devices were integrated into the existing LoRaWAN infrastructure at Curtin University with data collected and stored on a remote server. Case studies were performed for the City of Melville to address the suitability for such a system to provide insights into heat islands and urban forests. Testing was completed on the Curtin University campus, replicating the climate conditions, asset types and, dense building and tree environment found in the City of Melville.