OpenRaiser/Intern-Atlas huggingface dataset:
BERT Transformer model for Detecting Arabic GPT2 Auto-Generated Tweets
Venue: arXiv cs.CL 2021
Year: 2021

During the last two decades, we have progressively turned to the Internet and social media to find news, entertain conversations and share opinion. Recently, OpenAI has developed a ma-chine learning system called GPT-2 for Generative Pre-trained Transformer-2, which can pro-duce deepfake texts. It can generate blocks of text based on brief writing prompts that look like they were written by humans, facilitating the spread false or auto-generated text. In line with this progress, and in order to counteract potential dangers, several methods have been pro-posed for detecting text written by these language models. In this paper, we propose a transfer learning based model that will be able to detect if an Arabic sentence is written by humans or automatically generated by bots. Our dataset is based on tweets from a previous work, which we have crawled and extended using the Twitter API. We used GPT2-Small-Arabic to generate fake Arabic Sentences. For evaluation, we compared different recurrent neural network (RNN) word embeddings based baseline models, namely: LSTM, BI-LSTM, GRU and BI-GRU, with a transformer-based model. Our new transfer-learning model has obtained an accuracy up to 98%. To the best of our knowledge, this work is the first study where ARABERT and GPT2 were combined to detect and classify the Arabic auto-generated texts.
Machine Learning Based Early Fire Detection System using a Low-Cost Drone
Venue: arXiv cs.RO 2021
Year: 2021

This paper proposes a new machine learning based system for forest fire earlier detection in a low-cost and accurate manner. Accordingly, it is aimed to bring a new and definite perspective to visual detection in forest fires. A drone is constructed for this purpose. The microcontroller in the system has been programmed by training with deep learning methods, and the unmanned aerial vehicle has been given the ability to recognize the smoke, the earliest sign of fire detection. The common problem in the prevalent algorithms used in fire detection is the high false alarm and overlook rates. Confirming the result obtained from the visualization with an additional supervision stage will increase the reliability of the system as well as guarantee the accuracy of the result. Due to the mobile vision ability of the unmanned aerial vehicle, the data can be controlled from any point of view clearly and continuously. System performance are validated by conducting experiments in both simulation and physical environments.
Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection
Venue: arXiv cs.CL 2021
Year: 2021

Lexical semantic change detection is a new and innovative research field. The optimal fine-tuning of models including pre- and post-processing is largely unclear. We optimize existing models by (i) pre-training on large corpora and refining on diachronic target corpora tackling the notorious small data problem, and (ii) applying post-processing transformations that have been shown to improve performance on synchronic tasks. Our results provide a guide for the application and optimization of lexical semantic change detection models across various learning scenarios.
Gaussian Process-Based Model Predictive Control for Overtaking
Venue: arXiv cs.RO 2021
Year: 2021

This paper proposes a novel framework for addressing the challenge of autonomous overtaking and obstacle avoidance, which incorporates the overtaking path planning into Gaussian Process-based model predictive control (GPMPC). Compared with the conventional control strategies, this approach has two main advantages. Firstly, combining Gaussian Process (GP) regression with a nominal model allows for learning from model mismatch and unmodeled dynamics, which enhances a simple model and delivers significantly better results. Due to the approximation for propagating uncertainties, we can furthermore satisfy the constraints and thereby safety of the vehicle is ensured. Secondly, we convert the geometric relationship between the ego vehicle and other obstacle vehicles into the constraints. Without relying on a higherlevel path planner, this approach substantially reduces the computational burden. In addition, we transform the state constraints under the model predictive control (MPC) framework into a soft constraint and incorporate it as relaxed barrier function into the cost function, which makes the optimizer more efficient. Simulation results reveal the usefulness of the proposed approach.
Online Adversarial Purification based on Self-Supervision
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks are known to be vulnerable to adversarial examples, where a perturbation in the input space leads to an amplified shift in the latent network representation. In this paper, we combine canonical supervised learning with self-supervised representation learning, and present Self-supervised Online Adversarial Purification (SOAP), a novel defense strategy that uses a self-supervised loss to purify adversarial examples at test-time. Our approach leverages the label-independent nature of self-supervised signals and counters the adversarial perturbation with respect to the self-supervised tasks. SOAP yields competitive robust accuracy against state-of-the-art adversarial training and purification methods, with considerably less training complexity. In addition, our approach is robust even when adversaries are given knowledge of the purification defense strategy. To the best of our knowledge, our paper is the first that generalizes the idea of using self-supervised signals to perform online test-time purification.
Calculating a backtracking algorithm: an exercise in monadic program derivation
Venue: arXiv cs.PL 2021
Year: 2021

Equational reasoning is among the most important tools that functional programming provides us. Curiously, relatively less attention has been paid to reasoning about monadic programs. In this report we derive a backtracking algorithm for problem specifications that use a monadic unfold to generate possible solutions, which are filtered using a $\mathit{scanl}$-like predicate. We develop theorems that convert a variation of $\mathit{scanl}$ to a $\mathit{foldr}$ that uses the state monad, as well as theorems constructing hylomorphism. The algorithm is used to solve the $n$-queens puzzle, our running example. The aim is to develop theorems and patterns useful for the derivation of monadic programs, focusing on the intricate interaction between state and non-determinism.
Exploiting Web Images for Fine-Grained Visual Recognition by Eliminating Noisy Samples and Utilizing Hard Ones
Venue: arXiv cs.CV 2021
Year: 2021

Labeling objects at a subordinate level typically requires expert knowledge, which is not always available when using random annotators. As such, learning directly from web images for fine-grained recognition has attracted broad attention. However, the presence of label noise and hard examples in web images are two obstacles for training robust fine-grained recognition models. Therefore, in this paper, we propose a novel approach for removing irrelevant samples from real-world web images during training, while employing useful hard examples to update the network. Thus, our approach can alleviate the harmful effects of irrelevant noisy web images and hard examples to achieve better performance. Extensive experiments on three commonly used fine-grained datasets demonstrate that our approach is far superior to current state-of-the-art web-supervised methods.
Hierarchical Variational Auto-Encoding for Unsupervised Domain Generalization
Venue: arXiv cs.LG 2021
Year: 2021

We address the task of domain generalization, where the goal is to train a predictive model such that it is able to generalize to a new, previously unseen domain. We choose a hierarchical generative approach within the framework of variational autoencoders and propose a domain-unsupervised algorithm that is able to generalize to new domains without domain supervision. We show that our method is able to learn representations that disentangle domain-specific information from class-label specific information even in complex settings where domain structure is not observed during training. Our interpretable method outperforms previously proposed generative algorithms for domain generalization as well as other non-generative state-of-the-art approaches in several hierarchical domain settings including sequential overlapped near continuous domain shift. It also achieves competitive performance on the standard domain generalization benchmark dataset PACS compared to state-of-the-art approaches which rely on observing domain-specific information during training, as well as another domain unsupervised method. Additionally, we proposed model selection purely based on Evidence Lower Bound (ELBO) and also proposed weak domain supervision where implicit domain information can be added into the algorithm.
Reproducibility, Replicability and Beyond: Assessing Production Readiness of Aspect Based Sentiment Analysis in the Wild
Venue: arXiv cs.CL 2021
Year: 2021

With the exponential growth of online marketplaces and user-generated content therein, aspect-based sentiment analysis has become more important than ever. In this work, we critically review a representative sample of the models published during the past six years through the lens of a practitioner, with an eye towards deployment in production. First, our rigorous empirical evaluation reveals poor reproducibility: an average 4-5% drop in test accuracy across the sample. Second, to further bolster our confidence in empirical evaluation, we report experiments on two challenging data slices, and observe a consistent 12-55% drop in accuracy. Third, we study the possibility of transfer across domains and observe that as little as 10-25% of the domain-specific training dataset, when used in conjunction with datasets from other domains within the same locale, largely closes the gap between complete cross-domain and complete in-domain predictive performance. Lastly, we open-source two large-scale annotated review corpora from a large e-commerce portal in India in order to aid the study of replicability and transfer, with the hope that it will fuel further growth of the field.
Sequence-based Dynamic Handwriting Analysis for Parkinson's Disease Detection with One-dimensional Convolutions and BiGRUs
Venue: arXiv cs.CV 2021
Year: 2021

Parkinson's disease (PD) is commonly characterized by several motor symptoms, such as bradykinesia, akinesia, rigidity, and tremor. The analysis of patients' fine motor control, particularly handwriting, is a powerful tool to support PD assessment. Over the years, various dynamic attributes of handwriting, such as pen pressure, stroke speed, in-air time, etc., which can be captured with the help of online handwriting acquisition tools, have been evaluated for the identification of PD. Motion events, and their associated spatio-temporal properties captured in online handwriting, enable effective classification of PD patients through the identification of unique sequential patterns. This paper proposes a novel classification model based on one-dimensional convolutions and Bidirectional Gated Recurrent Units (BiGRUs) to assess the potential of sequential information of handwriting in identifying Parkinsonian symptoms. One-dimensional convolutions are applied to raw sequences as well as derived features; the resulting sequences are then fed to BiGRU layers to achieve the final classification. The proposed method outperformed state-of-the-art approaches on the PaHaW dataset and achieved competitive results on the NewHandPD dataset.
Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanations
Venue: arXiv cs.LG 2021
Year: 2021

Feature attribution is widely used in interpretable machine learning to explain how influential each measured input feature value is for an output inference. However, measurements can be uncertain, and it is unclear how the awareness of input uncertainty can affect the trust in explanations. We propose and study two approaches to help users to manage their perception of uncertainty in a model explanation: 1) transparently show uncertainty in feature attributions to allow users to reflect on, and 2) suppress attribution to features with uncertain measurements and shift attribution to other features by regularizing with an uncertainty penalty. Through simulation experiments, qualitative interviews, and quantitative user evaluations, we identified the benefits of moderately suppressing attribution uncertainty, and concerns regarding showing attribution uncertainty. This work adds to the understanding of handling and communicating uncertainty for model interpretability.
Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art
Venue: arXiv cs.LG 2021
Year: 2021

Safe learning and optimization deals with learning and optimization problems that avoid, as much as possible, the evaluation of non-safe input points, which are solutions, policies, or strategies that cause an irrecoverable loss (e.g., breakage of a machine or equipment, or life threat). Although a comprehensive survey of safe reinforcement learning algorithms was published in 2015, a number of new algorithms have been proposed thereafter, and related works in active learning and in optimization were not considered. This paper reviews those algorithms from a number of domains including reinforcement learning, Gaussian process regression and classification, evolutionary algorithms, and active learning. We provide the fundamental concepts on which the reviewed algorithms are based and a characterization of the individual algorithms. We conclude by explaining how the algorithms are connected and suggestions for future research.
Is academia becoming more localised? The growth of regional knowledge networks within international research collaboration
Venue: arXiv cs.SI 2021
Year: 2021

It is well-established that the process of learning and capability building is core to economic development and structural transformation. Since knowledge is `sticky', a key component of this process is learning-by-doing, which can be achieved via a variety of mechanisms including international research collaboration. Uncovering significant inter-country research ties using Scopus co-authorship data, we show that within-region collaboration has increased over the past five decades relative to international collaboration. Further supporting this insight, we find that while communities present in the global collaboration network before 2000 were often based on historical geopolitical or colonial lines, in more recent years they increasingly align with a simple partition of countries by regions. These findings are unexpected in light of a presumed continual increase in globalisation, and have significant implications for the design of programmes aimed at promoting international research collaboration and knowledge diffusion.
A Dual-arm Robot that Autonomously Lifts Up and Tumbles Heavy Plates Using Crane Pulley Blocks
Venue: arXiv cs.RO 2021
Year: 2021

This paper develops a planner that plans the action sequences and motion for a dual-arm robot to lift up and flip heavy plates using crane pulley blocks. The problem is motivated by the low payload of modern collaborative robots. Instead of directly manipulating heavy plates that collaborative robots cannot afford, the paper develops a planner for collaborative robots to operate crane pulley blocks. The planner assumes a target plate is pre-attached to the crane hook. It optimizes dual-arm action sequences and plans the robot's dual-arm motion that pulls the rope of the crane pulley blocks to lift up the plate. The crane pulley blocks reduce the payload that each robotic arm needs to bear. When the plate is lifted up to a satisfying pose, the planner plans a pushing motion for one of the robot arms to tumble over the plate while considering force and moment constraints. The article presents the technical details of the planner and several experiments and analysis carried out using a dual-arm robot made by two Universal Robots UR3 arms. The influence of various parameters and optimization goals are investigated and compared in depth. The results show that the proposed planner is flexible and efficient.
Hard satisfiable formulas for DPLL algorithms using heuristics with small memory
Venue: arXiv cs.CC 2021
Year: 2021

DPLL algorithm for solving the Boolean satisfiability problem (SAT) can be represented in the form of a procedure that, using heuristics $A$ and $B$, select the variable $x$ from the input formula $φ$ and the value $b$ and runs recursively on the formulas $φ[x := b]$ and $φ[x := 1 - b]$. Exponential lower bounds on the running time of DPLL algorithms on unsatisfiable formulas follow from the lower bounds for tree-like resolution proofs. Lower bounds on satisfiable formulas are also known for some classes of DPLL algorithms such as "myopic" and "drunken" algorithms. All lower bounds are made for the classes of DPLL algorithms that limit heuristics access to the formula. In this paper we consider DPLL algorithms with heuristics that have unlimited access to the formula but use small memory. We show that for any pair of heuristics with small memory there exists a family of satisfiable formulas $Φ_n$ such that a DPLL algorithm that uses these heuristics runs in exponential time on the formulas $Φ_n$.
Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
Venue: arXiv cs.CV 2021
Year: 2021

Rehearsal is a critical component for class-incremental continual learning, yet it requires a substantial memory budget. Our work investigates whether we can significantly reduce this memory budget by leveraging unlabeled data from an agent's environment in a realistic and challenging continual learning paradigm. Specifically, we explore and formalize a novel semi-supervised continual learning (SSCL) setting, where labeled data is scarce yet non-i.i.d. unlabeled data from the agent's environment is plentiful. Importantly, data distributions in the SSCL setting are realistic and therefore reflect object class correlations between, and among, the labeled and unlabeled data distributions. We show that a strategy built on pseudo-labeling, consistency regularization, Out-of-Distribution (OoD) detection, and knowledge distillation reduces forgetting in this setting. Our approach, DistillMatch, increases performance over the state-of-the-art by no less than 8.7% average task accuracy and up to 54.5% average task accuracy in SSCL CIFAR-100 experiments. Moreover, we demonstrate that DistillMatch can save up to 0.23 stored images per processed unlabeled image compared to the next best method which only saves 0.08. Our results suggest that focusing on realistic correlated distributions is a significantly new perspective, which accentuates the importance of leveraging the world's structure as a continual learning strategy.
Network-Agnostic Knowledge Transfer for Medical Image Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Conventional transfer learning leverages weights of pre-trained networks, but mandates the need for similar neural architectures. Alternatively, knowledge distillation can transfer knowledge between heterogeneous networks but often requires access to the original training data or additional generative networks. Knowledge transfer between networks can be improved by being agnostic to the choice of network architecture and reducing the dependence on original training data. We propose a knowledge transfer approach from a teacher to a student network wherein we train the student on an independent transferal dataset, whose annotations are generated by the teacher. Experiments were conducted on five state-of-the-art networks for semantic segmentation and seven datasets across three imaging modalities. We studied knowledge transfer from a single teacher, combination of knowledge transfer and fine-tuning, and knowledge transfer from multiple teachers. The student model with a single teacher achieved similar performance as the teacher; and the student model with multiple teachers achieved better performance than the teachers. The salient features of our algorithm include: 1)no need for original training data or generative networks, 2) knowledge transfer between different architectures, 3) ease of implementation for downstream tasks by using the downstream task dataset as the transferal dataset, 4) knowledge transfer of an ensemble of models, trained independently, into one student model. Extensive experiments demonstrate that the proposed algorithm is effective for knowledge transfer and easily tunable.
Decentralized and Online Coded Caching with Shared Caches: Fundamental Limits with Uncoded Prefetching
Venue: arXiv cs.IT 2021
Year: 2021

Decentralized coded caching scheme, introduced by Maddah-Ali and Niesen, assumes that the caches are filled with no coordination. This work identifies a decentralized coded caching scheme -- under the assumption of uncoded placement -- for shared cache network, where each cache serves multiple users. Each user has access to only a single cache and the number of caches is less than or equal to the number of users. For this setting, we derive the optimal worst-case delivery time for any user-to-cache association profile where each such profile describes the number of users served by each cache. The optimality is shown using an index-coding based converse. Further, we improve the delivery scheme to accommodate redundant demands. Also, an optimal linear error correcting delivery scheme is proposed for the worst-case demand scenario. Next, we consider the Least Recently Sent (LRS) online coded caching scheme where the caches need to be updated based on the sequence of demands made by the users. Cache update happens if any of the demanded file was not partially cached at the users. The update is done by replacing the least recently sent file with the new file. But, the least recently sent file need not be unique. In that case, there needs to be some ordering of the files which are getting partially cached, or else centralized coordination would have to be assumed which does not exist. If each user removes any of the least recently used files at random, then the next delivery phase will not serve the purpose. A modification is suggested for the scheme by incorporating an ordering of files. Moreover, all the above results with shared caches are extended to the online setting.
Digital Transformations of Classrooms in Virtual Reality
Venue: arXiv cs.HC 2021
Year: 2021

With rapid developments in consumer-level head-mounted displays and computer graphics, immersive VR has the potential to take online and remote learning closer to real-world settings. However, the effects of such digital transformations on learners, particularly for VR, have not been evaluated in depth. This work investigates the interaction-related effects of sitting positions of learners, visualization styles of peer-learners and teachers, and hand-raising behaviors of virtual peer-learners on learners in an immersive VR classroom, using eye tracking data. Our results indicate that learners sitting in the back of the virtual classroom may have difficulties extracting information. Additionally, we find indications that learners engage with lectures more efficiently if virtual avatars are visualized with realistic styles. Lastly, we find different eye movement behaviors towards different performance levels of virtual peer-learners, which should be investigated further. Our findings present an important baseline for design decisions for VR classrooms.