OpenRaiser/Intern-Atlas huggingface dataset:
Multi-class Probabilistic Bounds for Self-learning
Venue: arXiv cs.LG 2021
Year: 2021

Self-learning is a classical approach for learning with both labeled and unlabeled observations which consists in giving pseudo-labels to unlabeled training instances with a confidence score over a predetermined threshold. At the same time, the pseudo-labeling technique is prone to error and runs the risk of adding noisy labels into unlabeled training data. In this paper, we present a probabilistic framework for analyzing self-learning in the multi-class classification scenario with partially labeled data. First, we derive a transductive bound over the risk of the multi-class majority vote classifier. Based on this result, we propose to automatically choose the threshold for pseudo-labeling that minimizes the transductive bound. Then, we introduce a mislabeling error model to analyze the error of the majority vote classifier in the case of the pseudo-labeled data. We derive a probabilistic C-bound over the majority vote error when an imperfect label is given. Empirical results on different data sets show the effectiveness of our framework compared to several state-of-the-art semi-supervised approaches.
Overview of the Arabic Sentiment Analysis 2021 Competition at KAUST
Venue: arXiv cs.CL 2021
Year: 2021

This paper provides an overview of the Arabic Sentiment Analysis Challenge organized by King Abdullah University of Science and Technology (KAUST). The task in this challenge is to develop machine learning models to classify a given tweet into one of the three categories Positive, Negative, or Neutral. From our recently released ASAD dataset, we provide the competitors with 55K tweets for training, and 20K tweets for validation, based on which the performance of participating teams are ranked on a leaderboard, https://www.kaggle.com/c/arabic-sentiment-analysis-2021-kaust. The competition received in total 1247 submissions from 74 teams (99 team members). The final winners are determined by another private set of 20K tweets that have the same distribution as the training and validation set. In this paper, we present the main findings in the competition and summarize the methods and tools used by the top ranked teams. The full dataset of 100K labeled tweets is also released for public usage, at https://www.kaggle.com/c/arabic-sentiment-analysis-2021-kaust/data.
Deep Spatio-Temporal Wind Power Forecasting
Venue: arXiv cs.LG 2021
Year: 2021

Wind power forecasting has drawn increasing attention among researchers as the consumption of renewable energy grows. In this paper, we develop a deep learning approach based on encoder-decoder structure. Our model forecasts wind power generated by a wind turbine using its spatial location relative to other turbines and historical wind speed data. In this way, we effectively integrate spatial dependency and temporal trends to make turbine-specific predictions. The advantages of our method over existing work can be summarized as 1) it directly predicts wind power based on historical wind speed, without the need for prediction of wind speed first, and then using a transformation; 2) it can effectively capture long-term dependency 3) our model is more scalable and efficient compared with other deep learning based methods. We demonstrate the efficacy of our model on the benchmark real-world datasets.
Robust Temporal Ensembling for Learning with Noisy Labels
Venue: arXiv cs.CV 2021
Year: 2021

Successful training of deep neural networks with noisy labels is an essential capability as most real-world datasets contain some amount of mislabeled data. Left unmitigated, label noise can sharply degrade typical supervised learning approaches. In this paper, we present robust temporal ensembling (RTE), which combines robust loss with semi-supervised regularization methods to achieve noise-robust learning. We demonstrate that RTE achieves state-of-the-art performance across the CIFAR-10, CIFAR-100, ImageNet, WebVision, and Food-101N datasets, while forgoing the recent trend of label filtering and/or fixing. Finally, we show that RTE also retains competitive corruption robustness to unforeseen input noise using CIFAR-10-C, obtaining a mean corruption error (mCE) of 13.50% even in the presence of an 80% noise ratio, versus 26.9% mCE with standard methods on clean data.
A secure home automation prototype built on raspberry-pi
Venue: arXiv cs.RO 2021
Year: 2021

With the development of sensors, wireless mobile communication, embedded system, the technologies of the Internet of Things have been widely used in SmartMeter, public security, intelligent building and so on. Because of its huge market prospects, the Internet of Things has been paid close attention by several governments all over the world. IoT facilitates the seamless integration of wireless sensor networks. In this paper, we present an IoT prototype that is built on Raspberry Pi and uses SMTP (simple mail transfer protocol) for communication. Through this device, we have proposed a communication system that is less complex and more secure. It integrates with any "thing" and makes it electronically communicable. We give an implementation of the prototyping system and system validation.
An Empirical Study of Accuracy, Fairness, Explainability, Distributional Robustness, and Adversarial Robustness
Venue: arXiv cs.LG 2021
Year: 2021

To ensure trust in AI models, it is becoming increasingly apparent that evaluation of models must be extended beyond traditional performance metrics, like accuracy, to other dimensions, such as fairness, explainability, adversarial robustness, and distribution shift. We describe an empirical study to evaluate multiple model types on various metrics along these dimensions on several datasets. Our results show that no particular model type performs well on all dimensions, and demonstrate the kinds of trade-offs involved in selecting models evaluated along multiple dimensions.
Mitigation of Adversarial Policy Imitation via Constrained Randomization of Policy (CRoP)
Venue: arXiv cs.LG 2021
Year: 2021

Deep reinforcement learning (DRL) policies are vulnerable to unauthorized replication attacks, where an adversary exploits imitation learning to reproduce target policies from observed behavior. In this paper, we propose Constrained Randomization of Policy (CRoP) as a mitigation technique against such attacks. CRoP induces the execution of sub-optimal actions at random under performance loss constraints. We present a parametric analysis of CRoP, address the optimality of CRoP, and establish theoretical bounds on the adversarial budget and the expectation of loss. Furthermore, we report the experimental evaluation of CRoP in Atari environments under adversarial imitation, which demonstrate the efficacy and feasibility of our proposed method against policy replication attacks.
Fundamental Rate-Memory Tradeoff for Coded Caching in Presence of User Inactivity
Venue: arXiv cs.IT 2021
Year: 2021

Coded caching utilizes proper file subpacketization and coded delivery to make full use of the multicast opportunities in content delivery, to alleviate file transfer load in massive content delivery scenarios. Most existing work considers deterministic environments. An important practical topic is to characterize the impact of the uncertainty from user inactivity on coded caching. We consider a one server cache-enabled network under homogeneous file and network settings in presence of user inactivity. Unlike random or probabilistic caching studied in the literature, deterministic coded caching is considered, with the objective to minimize the worst-case backhaul load by optimizing the file subpacketization and the caching strategy. First, a coded caching method is used, where each file is split into the same type of fragments labeled using sets with fixed cardinality, and the optimality of the selected cardinality is proved. Optimal file subpacketization by splitting the file into multiple types of fragments labeled with multiple cardinalities is then discussed. We show that the closed-form optimum turns out to be given by a fixed cardinality -- optimizing for user inactivity only affects file delivery, cache placement is not affected. A decentralized version is also discussed and analyzed, where each user fills its storage independently at random without centralized coordination, and user inactivity is taken into account in file delivery. Simulation results show that the optimization based centralized coded caching scheme provides performance comparable to the ideal scenario assuming full knowledge of user inactivity in the placement phase, while decentralized caching performs slightly worse against user inactivity.
Vision-Aided Beam Tracking: Explore the Proper Use of Camera Images with Deep Learning
Venue: arXiv cs.LG 2021
Year: 2021

We investigate the problem of wireless beam tracking on mmWave bands with the assistance of camera images. In particular, based on the user's beam indices used and camera images taken in the trajectory, we predict the optimal beam indices in the next few time spots. To resolve this problem, we first reformulate the "ViWi" dataset in [1] to get rid of the image repetition problem. Then we develop a deep learning approach and investigate various model components to achieve the best performance. Finally, we explore whether, when, and how to use the image for better beam prediction. To answer this question, we split the dataset into three clusters -- (LOS, light NLOS, serious NLOS)-like -- based on the standard deviation of the beam sequence. With experiments we demonstrate that using the image indeed helps beam tracking especially when the user is in serious NLOS, and the solution relies on carefully-designed dataset for training a model. Generally speaking, including NLOS-like data for training a model does not benefit beam tracking of the user in LOS, but including light NLOS-like data for training a model benefits beam tracking of the user in serious NLOS.
LR-to-HR Face Hallucination with an Adversarial Progressive Attribute-Induced Network
Venue: arXiv cs.CV 2021
Year: 2021

Face super-resolution is a challenging and highly ill-posed problem since a low-resolution (LR) face image may correspond to multiple high-resolution (HR) ones during the hallucination process and cause a dramatic identity change for the final super-resolved results. Thus, to address this problem, we propose an end-to-end progressive learning framework incorporating facial attributes and enforcing additional supervision from multi-scale discriminators. By incorporating facial attributes into the learning process and progressively resolving the facial image, the mapping between LR and HR images is constrained more, and this significantly helps to reduce the ambiguity and uncertainty in one-to-many mapping. In addition, we conduct thorough evaluations on the CelebA dataset following the settings of previous works (i.e. super-resolving by a factor of 8x from tiny 16x16 face images.), and the results demonstrate that the proposed approach can yield satisfactory face hallucination images outperforming other state-of-the-art approaches.
Classifying Tweet Sentiment Using the Hidden State and Attention Matrix of a Fine-tuned BERTweet Model
Venue: arXiv cs.CL 2021
Year: 2021

This paper introduces a study on tweet sentiment classification. Our task is to classify a tweet as either positive or negative. We approach the problem in two steps, namely embedding and classifying. Our baseline methods include several combinations of traditional embedding methods and classification algorithms. Furthermore, we explore the current state-of-the-art tweet analysis model, BERTweet, and propose a novel approach in which features are engineered from the hidden states and attention matrices of the model, inspired by empirical study of the tweets. Using a multi-layer perceptron trained with a high dropout rate for classification, our proposed approach achieves a validation accuracy of 0.9111.
Subdimensional Expansion Using Attention-Based Learning For Multi-Agent Path Finding
Venue: arXiv cs.AI 2021
Year: 2021

Multi-Agent Path Finding (MAPF) finds conflict-free paths for multiple agents from their respective start to goal locations. MAPF is challenging as the joint configuration space grows exponentially with respect to the number of agents. Among MAPF planners, search-based methods, such as CBS and M*, effectively bypass the curse of dimensionality by employing a dynamically-coupled strategy: agents are planned in a fully decoupled manner at first, where potential conflicts between agents are ignored; and then agents either follow their individual plans or are coupled together for planning to resolve the conflicts between them. In general, the number of conflicts to be resolved decides the run time of these planners and most of the existing work focuses on how to efficiently resolve these conflicts. In this work, we take a different view and aim to reduce the number of conflicts (and thus improve the overall search efficiency) by improving each agent's individual plan. By leveraging a Visual Transformer, we develop a learning-based single-agent planner, which plans for a single agent while paying attention to both the structure of the map and other agents with whom conflicts may happen. We then develop a novel multi-agent planner called LM* by integrating this learning-based single-agent planner with M*. Our results show that for both "seen" and "unseen" maps, in comparison with M*, LM* has fewer conflicts to be resolved and thus, runs faster and enjoys higher success rates. We empirically show that MAPF solutions computed by LM* are near-optimal. Our code is available at https://github.com/lakshayvirmani/learning-assisted-mstar .
Time-Distributed Feature Learning in Network Traffic Classification for Internet of Things
Venue: arXiv cs.NI 2021
Year: 2021

The plethora of Internet of Things (IoT) devices leads to explosive network traffic. The network traffic classification (NTC) is an essential tool to explore behaviours of network flows, and NTC is required for Internet service providers (ISPs) to manage the performance of the IoT network. We propose a novel network data representation, treating the traffic data as a series of images. Thus, the network data is realized as a video stream to employ time-distributed (TD) feature learning. The intra-temporal information within the network statistical data is learned using convolutional neural networks (CNN) and long short-term memory (LSTM), and the inter pseudo-temporal feature among the flows is learned by TD multi-layer perceptron (MLP). We conduct experiments using a large data-set with more number of classes. The experimental result shows that the TD feature learning elevates the network classification performance by 10%.
USIS: Unsupervised Semantic Image Synthesis
Venue: arXiv cs.CV 2021
Year: 2021

Semantic Image Synthesis (SIS) is a subclass of image-to-image translation where a photorealistic image is synthesized from a segmentation mask. SIS has mostly been addressed as a supervised problem. However, state-of-the-art methods depend on a huge amount of labeled data and cannot be applied in an unpaired setting. On the other hand, generic unpaired image-to-image translation frameworks underperform in comparison, because they color-code semantic layouts and feed them to traditional convolutional networks, which then learn correspondences in appearance instead of semantic content. In this initial work, we propose a new Unsupervised paradigm for Semantic Image Synthesis (USIS) as a first step towards closing the performance gap between paired and unpaired settings. Notably, the framework deploys a SPADE generator that learns to output images with visually separable semantic classes using a self-supervised segmentation loss. Furthermore, in order to match the color and texture distribution of real images without losing high-frequency information, we propose to use whole image wavelet-based discrimination. We test our methodology on 3 challenging datasets and demonstrate its ability to generate multimodal photorealistic images with an improved quality in the unpaired setting.
Conceptual Data Modeling: Entity-Relationship Models as Thinging Machines
Venue: arXiv cs.DB 2021
Year: 2021

Data modeling is a process of developing a model to design and develop a data system that supports an organization s various business processes. A conceptual data model represents a technology-independent specification of structure of data to be stored within a database. The model aims at providing richer expressiveness and incorporating a set of semantics to (a) support the design, control, and integrity parts of the data stored in data management structures and (b) coordinate viewing of connections and ideas on a database. The described structure of the data is often represented in an entity-relationship (ER) model, which was one of the first data-modeling techniques and is likely to continue to be a popular way of characterizing entity classes, attributes and relationships. This paper is an attempt to examine the basic ER modeling notions to analyze the concepts to which they refer as well as ways to represent them. In such a mission, we apply a new modeling methodology (thinging machine; TM) to ER in terms of its fundamental building constructs, representation entities, relationships and attributes. The goal of this venture is to further the understanding of data models and enrich their semantics. Three specific contributions to modeling in this context are incorporated: (a) using the TM model s five generic actions to inject processing in the ER structure; (b) relating the single ontological element of TM modeling (i.e., a thing/machine or thimac) to ER entities and relationships; and (c) proposing a high-level integrated, extended ER model that includes structural and time-oriented notions (e.g., events or behavior).
Dr Jekyll and Mr Hyde: the Strange Case of Off-Policy Policy Updates
Venue: arXiv cs.LG 2021
Year: 2021

The policy gradient theorem states that the policy should only be updated in states that are visited by the current policy, which leads to insufficient planning in the off-policy states, and thus to convergence to suboptimal policies. We tackle this planning issue by extending the policy gradient theory to policy updates with respect to any state density. Under these generalized policy updates, we show convergence to optimality under a necessary and sufficient condition on the updates' state densities, and thereby solve the aforementioned planning issue. We also prove asymptotic convergence rates that significantly improve those in the policy gradient literature. To implement the principles prescribed by our theory, we propose an agent, Dr Jekyll & Mr Hyde (JH), with a double personality: Dr Jekyll purely exploits while Mr Hyde purely explores. JH's independent policies allow to record two separate replay buffers: one on-policy (Dr Jekyll's) and one off-policy (Mr Hyde's), and therefore to update JH's models with a mixture of on-policy and off-policy updates. More than an algorithm, JH defines principles for actor-critic algorithms to satisfy the requirements we identify in our analysis. We extensively test on finite MDPs where JH demonstrates a superior ability to recover from converging to a suboptimal policy without impairing its speed of convergence. We also implement a deep version of the algorithm and test it on a simple problem where it shows promising results.
Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning
Venue: arXiv cs.CV 2021
Year: 2021

A convolutional neural network (ConvNet) is usually trained and then tested using images drawn from the same distribution. To generalize a ConvNet to various tasks often requires a complete training dataset that consists of images drawn from different tasks. In most scenarios, it is nearly impossible to collect every possible representative dataset as a priori. The new data may only become available after the ConvNet is deployed in clinical practice. ConvNet, however, may generate artifacts on out-of-distribution testing samples. In this study, we present Targeted Gradient Descent (TGD), a novel fine-tuning method that can extend a pre-trained network to a new task without revisiting data from the previous task while preserving the knowledge acquired from previous training. To a further extent, the proposed method also enables online learning of patient-specific data. The method is built on the idea of reusing a pre-trained ConvNet's redundant kernels to learn new knowledge. We compare the performance of TGD to several commonly used training approaches on the task of Positron emission tomography (PET) image denoising. Results from clinical images show that TGD generated results on par with training-from-scratch while significantly reducing data preparation and network training time. More importantly, it enables online learning on the testing study to enhance the network's generalization capability in real-world applications.
Como o Game Design Pode Incentivar o Comportamento Tóxico em Jogos Online
Venue: arXiv cs.HC 2021
Year: 2021

Toxic behavior is one of the problems most associated with the gaming community. Reports of hate speech and anti-gaming behavior are common in many online multiplayer games. Many studies address this area, focusing on players and, principally, on the content and its negative impact on the community. However, it is possible that other factors influence the toxicity. In this article, we investigate how game design elements influence toxic behavior, using toxic disinhibition factors as a basis for the analysis. For this, four of the main games of the MOBA genre were analyzed with the support of the MDA Framework and a questionnaire with players. The results show that most players suffered offenses and offended other people in the game, in addition to recognizing MOBA as the genre that generates the most impatience among them. Furthermore, we raise and discuss different elements and mechanics in games that impact the well-being of the community through the individualization of the player, elimination of solidarities, and affirmation of indifference towards toxic behavior. With this, we hope to help game developers and communities to promote integration dynamics.
Phonetic Word Embeddings
Venue: arXiv cs.CL 2021
Year: 2021

This work presents a novel methodology for calculating the phonetic similarity between words taking motivation from the human perception of sounds. This metric is employed to learn a continuous vector embedding space that groups similar sounding words together and can be used for various downstream computational phonology tasks. The efficacy of the method is presented for two different languages (English, Hindi) and performance gains over previous reported works are discussed on established tests for predicting phonetic similarity. To address limited benchmarking mechanisms in this field, we also introduce a heterographic pun dataset based evaluation methodology to compare the effectiveness of acoustic similarity algorithms. Further, a visualization of the embedding space is presented with a discussion on the various possible use-cases of this novel algorithm. An open-source implementation is also shared to aid reproducibility and enable adoption in related tasks.