OpenRaiser/Intern-Atlas huggingface dataset:
Exploring Semantic Relationships for Unpaired Image Captioning
Venue: arXiv cs.CV 2021
Year: 2021

Recently, image captioning has aroused great interest in both academic and industrial worlds. Most existing systems are built upon large-scale datasets consisting of image-sentence pairs, which, however, are time-consuming to construct. In addition, even for the most advanced image captioning systems, it is still difficult to realize deep image understanding. In this work, we achieve unpaired image captioning by bridging the vision and the language domains with high-level semantic information. The motivation stems from the fact that the semantic concepts with the same modality can be extracted from both images and descriptions. To further improve the quality of captions generated by the model, we propose the Semantic Relationship Explorer, which explores the relationships between semantic concepts for better understanding of the image. Extensive experiments on MSCOCO dataset show that we can generate desirable captions without paired datasets. Furthermore, the proposed approach boosts five strong baselines under the paired setting, where the most significant improvement in CIDEr score reaches 8%, demonstrating that it is effective and generalizes well to a wide range of models.
FedXGBoost: Privacy-Preserving XGBoost for Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning is the distributed machine learning framework that enables collaborative training across multiple parties while ensuring data privacy. Practical adaptation of XGBoost, the state-of-the-art tree boosting framework, to federated learning remains limited due to high cost incurred by conventional privacy-preserving methods. To address the problem, we propose two variants of federated XGBoost with privacy guarantee: FedXGBoost-SMM and FedXGBoost-LDP. Our first protocol FedXGBoost-SMM deploys enhanced secure matrix multiplication method to preserve privacy with lossless accuracy and lower overhead than encryption-based techniques. Developed independently, the second protocol FedXGBoost-LDP is heuristically designed with noise perturbation for local differential privacy, and empirically evaluated on real-world and synthetic datasets.
Overall Behavioural Index (OBI) For Measuring Segregation
Venue: arXiv cs.SI 2021
Year: 2021

Segregation, defined as the degree of separation between two or more population groups, helps to understand a complex social environment and subsequently provides a basis for public policy intervention. To measure segregation, past works often propose indexes that are criticized for being over-simplified and over-reduced. In other words, these indexes use the highly aggregated information to measure segregation. In this paper, we propose three novel indexes to measure segregation, namely: (i) Individual Segregation Index (ISI), (ii) Individual Inclination Index (III), and (iii) Overall Behavioural Index (OBI). The ISI index measures individuals' segregation, and the III index reports the individuals' inclination towards other population groups. The OBI index, calculated using both III and ISI index, is non-simplified and not only recognizes individuals' connectivity behaviour but group's connectivity behavioural distribution as well. By considering commonly used Freeman's segregation and homophily index as baseline indexes, we compare the OBI index on real call data records (CDR) dataset of Estonia to show the effectiveness of the proposed indexes.
A Comprehensive Review on Non-Neural Networks Collaborative Filtering Recommendation Systems
Venue: arXiv cs.IR 2021
Year: 2021

Over the past two decades, recommender systems have attracted a lot of interest due to the explosion in the amount of data in online applications. A particular attention has been paid to collaborative filtering, which is the most widely used in applications that involve information recommendations. Collaborative filtering (CF) uses the known preference of a group of users to make predictions and recommendations about the unknown preferences of other users (recommendations are made based on the past behavior of users). First introduced in the 1990s, a wide variety of increasingly successful models have been proposed. Due to the success of machine learning techniques in many areas, there has been a growing emphasis on the application of such algorithms in recommendation systems. In this article, we present an overview of the CF approaches for recommender systems, their two main categories, and their evaluation metrics. We focus on the application of classical Machine Learning algorithms to CF recommender systems by presenting their evolution from their first use-cases to advanced Machine Learning models. We attempt to provide a comprehensive and comparative overview of CF systems (with python implementations) that can serve as a guideline for research and practice in this area.
Guiding vector fields in Paparazzi autopilot
Venue: arXiv cs.RO 2021
Year: 2021

This article is a technical report on the two different guidance systems based on vector fields that can be found in Paparazzi, a free sw/hw autopilot. Guiding vector fields allow autonomous vehicles to track paths described by the user mathematically. In particular, we allow two descriptions of the path with an implicit or a parametric function. Each description is associated with its corresponding guiding vector field algorithm. The implementations of the two algorithms are light enough to be run in a modern microcontroller. We will cover the basic theory on how they work, how a user can implement its own paths in Paparazzi, how to exploit them to coordinate multiple vehicles, and we finish with some experimental results. Although the presented implementation is focused on fixed-wing aircraft, the guidance is also applicable to other kinds of aerial vehicles such as rotorcraft.
Adversarial Attack on Graph Neural Networks as An Influence Maximization Problem
Venue: arXiv cs.LG 2021
Year: 2021

Graph neural networks (GNNs) have attracted increasing interests. With broad deployments of GNNs in real-world applications, there is an urgent need for understanding the robustness of GNNs under adversarial attacks, especially in realistic setups. In this work, we study the problem of attacking GNNs in a restricted and realistic setup, by perturbing the features of a small set of nodes, with no access to model parameters and model predictions. Our formal analysis draws a connection between this type of attacks and an influence maximization problem on the graph. This connection not only enhances our understanding on the problem of adversarial attack on GNNs, but also allows us to propose a group of effective and practical attack strategies. Our experiments verify that the proposed attack strategies significantly degrade the performance of three popular GNN models and outperform baseline adversarial attack strategies.
Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panorama
Venue: arXiv cs.CV 2021
Year: 2021

We present Omnidirectional Neural Radiance Fields (OmniNeRF), the first method to the application of parallax-enabled novel panoramic view synthesis. Recent works for novel view synthesis focus on perspective images with limited field-of-view and require sufficient pictures captured in a specific condition. Conversely, OmniNeRF can generate panorama images for unknown viewpoints given a single equirectangular image as training data. To this end, we propose to augment the single RGB-D panorama by projecting back and forth between a 3D world and different 2D panoramic coordinates at different virtual camera positions. By doing so, we are able to optimize an Omnidirectional Neural Radiance Field with visible pixels collecting from omnidirectional viewing angles at a fixed center for the estimation of new viewing angles from varying camera positions. As a result, the proposed OmniNeRF achieves convincing renderings of novel panoramic views that exhibit the parallax effect. We showcase the effectiveness of each of our proposals on both synthetic and real-world datasets.
FedCM: Federated Learning with Client-level Momentum
Venue: arXiv cs.LG 2021
Year: 2021

Federated Learning is a distributed machine learning approach which enables model training without data sharing. In this paper, we propose a new federated learning algorithm, Federated Averaging with Client-level Momentum (FedCM), to tackle problems of partial participation and client heterogeneity in real-world federated learning applications. FedCM aggregates global gradient information in previous communication rounds and modifies client gradient descent with a momentum-like term, which can effectively correct the bias and improve the stability of local SGD. We provide theoretical analysis to highlight the benefits of FedCM. We also perform extensive empirical studies and demonstrate that FedCM achieves superior performance in various tasks and is robust to different levels of client numbers, participation rate and client heterogeneity.
An End-to-End Khmer Optical Character Recognition using Sequence-to-Sequence with Attention
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents an end-to-end deep convolutional recurrent neural network solution for Khmer optical character recognition (OCR) task. The proposed solution uses a sequence-to-sequence (Seq2Seq) architecture with attention mechanism. The encoder extracts visual features from an input text-line image via layers of residual convolutional blocks and a layer of gated recurrent units (GRU). The features are encoded in a single context vector and a sequence of hidden states which are fed to the decoder for decoding one character at a time until a special end-of-sentence (EOS) token is reached. The attention mechanism allows the decoder network to adaptively select parts of the input image while predicting a target character. The Seq2Seq Khmer OCR network was trained on a large collection of computer-generated text-line images for seven common Khmer fonts. The proposed model's performance outperformed the state-of-art Tesseract OCR engine for Khmer language on the 3000-images test set by achieving a character error rate (CER) of 1% vs 3%.
Certifying a probabilistic parallel modular algorithm for rational univariate representation
Venue: arXiv cs.SC 2021
Year: 2021

This paper is about solving polynomial systems. It first recalls how to do that efficiently with a very high probability of correctness by reconstructing a rational univariate representation (rur) using Groebner revlex computation, Berlekamp-Massey algorithm and Hankel linear system solving modulo several primes in parallel. Then it introduces a new method (theorem \ref{prop:check}) for rur certification that is effective for most polynomial systems.These algorithms are implemented in https://www-fourier.univ-grenoble-alpes.fr/~parisse/giac.html since version 1.7.0-13 or 1.7.0-17 for certification, it has (July 2021) leading performances on multiple CPU, at least for an open-source software.
Leveraging Conditional Generative Models in a General Explanation Framework of Classifier Decisions
Venue: arXiv cs.LG 2021
Year: 2021

Providing a human-understandable explanation of classifiers' decisions has become imperative to generate trust in their use for day-to-day tasks. Although many works have addressed this problem by generating visual explanation maps, they often provide noisy and inaccurate results forcing the use of heuristic regularization unrelated to the classifier in question. In this paper, we propose a new general perspective of the visual explanation problem overcoming these limitations. We show that visual explanation can be produced as the difference between two generated images obtained via two specific conditional generative models. Both generative models are trained using the classifier to explain and a database to enforce the following properties: (i) All images generated by the first generator are classified similarly to the input image, whereas the second generator's outputs are classified oppositely. (ii) Generated images belong to the distribution of real images. (iii) The distances between the input image and the corresponding generated images are minimal so that the difference between the generated elements only reveals relevant information for the studied classifier. Using symmetrical and cyclic constraints, we present two different approximations and implementations of the general formulation. Experimentally, we demonstrate significant improvements w.r.t the state-of-the-art on three different public data sets. In particular, the localization of regions influencing the classifier is consistent with human annotations.
Neural Controlled Differential Equations for Online Prediction Tasks
Venue: arXiv cs.LG 2021
Year: 2021

Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in \textit{online prediction tasks}, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that interpolation schemes for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks.
BIM, Digital Twin and Cyber-Physical Systems: crossing and blurring boundaries
Venue: arXiv cs.CY 2021
Year: 2021

Digital Twin in construction and the built environment have started to attract the attention of researchers and practitioners in recent times. Its anticipated value proposition is focussed on its capability of generating new understanding and insights into an asset at all stages of its life cycle, exploiting diverse data sets from a multitude of sources and professions, in real or near real-time. However, there is still a significant debate about the delineation (i.e. communalities and differences) between digital twin and other related concepts, particularly Building Information Modelling (BIM) and Cyber-Physical Systems (CPS). To date, this debate has been confined to social media discussions, insights blogs and position papers. This paper addresses this challenge using a systematic review. The aim is to investigate communalities and differences between the three concepts, Digital Twin, BIM and CPS. The results of this paper are expected to foster the discussion around this theme within construction and the built environment.
Größe als Erfolgsgarant? Zur Bedeutung der Organisationstruktur für die Einwerbung von Drittmitteln der Deutschen Forschungsgemeinschaft
Venue: arXiv cs.CY 2021
Year: 2021

Research funding through third-party financing is of considerable importance for the German science system. The funds of the German Research Foundation (DFG) serve as the central focus due to the high reputation of the foundation. However, it has not been clarified yet to what extent the chances of successfully acquiring these funds depend on the structure of the university as an institution. The present study analyses DFG funding in the context of university research and examines the role of organisational conditions in the acquisition of funding. Several factors, such as size of the institution, equipment, and teaching activities, are analysed. The empirical study focuses on four subjects and investigates the correlation between funding success and conditional factors using a Bayesian approach. Results reveal the considerable relevance of the factors size as well as provision of academic and non-academic personnel. This implies that the organisational conditions are to be taken into account while evaluating third-party financing success.
Conditional Neural Relational Inference for Interacting Systems
Venue: arXiv cs.LG 2021
Year: 2021

In this work, we want to learn to model the dynamics of similar yet distinct groups of interacting objects. These groups follow some common physical laws that exhibit specificities that are captured through some vectorial description. We develop a model that allows us to do conditional generation from any such group given its vectorial description. Unlike previous work on learning dynamical systems that can only do trajectory completion and require a part of the trajectory dynamics to be provided as input in generation time, we do generation using only the conditioning vector with no access to generation time's trajectories. We evaluate our model in the setting of modeling human gait and, in particular pathological human gait.
Obstacle Detection for BVLOS Drones
Venue: arXiv cs.CV 2021
Year: 2021

With the introduction of new regulations in the European Union, the future of Beyond Visual Line Of Sight (BVLOS) drones is set to bloom. This led to the creation of the theBEAST project, which aims to create an autonomous security drone, with focus on those regulations and on safety. This technical paper describes the first steps of a module within this project, which revolves around detecting obstacles so they can be avoided in a fail-safe landing. A deep learning powered object detection method is the subject of our research, and various experiments are held to maximize its performance, such as comparing various data augmentation techniques or YOLOv3 and YOLOv5. According to the results of the experiments, we conclude that although object detection is a promising approach to resolve this problem, more volume of data is required for potential usage in a real-life application.
On decoding of a specific type of self-dual codes
Venue: arXiv cs.IT 2021
Year: 2021

This work introduces a decoding strategy for binary self-dual codes possessing an automorphism of a specific type. The proposed algorithm is a hard decision iterative decoding scheme. The enclosed experiments show that the new decoding concept performs error correction beyond the upper bound for the code correction capability. Moreover, we prove that the requirements for the new algorithm hold for any binary self-dual code having an automorphism of the specific type, which makes decoding of this large group of codes possible.
Automatic Plant Cover Estimation with Convolutional Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Monitoring the responses of plants to environmental changes is essential for plant biodiversity research. This, however, is currently still being done manually by botanists in the field. This work is very laborious, and the data obtained is, though following a standardized method to estimate plant coverage, usually subjective and has a coarse temporal resolution. To remedy these caveats, we investigate approaches using convolutional neural networks (CNNs) to automatically extract the relevant data from images, focusing on plant community composition and species coverages of 9 herbaceous plant species. To this end, we investigate several standard CNN architectures and different pretraining methods. We find that we outperform our previous approach at higher image resolutions using a custom CNN with a mean absolute error of 5.16%. In addition to these investigations, we also conduct an error analysis based on the temporal aspect of the plant cover images. This analysis gives insight into where problems for automatic approaches lie, like occlusion and likely misclassifications caused by temporal changes.
Abstract Geometrical Computation 11: Slanted Firing Squad Synchronisation on Signal Machines
Venue: arXiv cs.DM 2021
Year: 2021

Firing Squad Synchronisation on Cellular Automata is the dynamical synchronisation of finitely many cells without any prior knowledge of their range. This can be conceived as a signal with an infinite speed. Most of the proposed constructions naturally translate to the continuous setting of signal machines and generate fractal figures with an accumulation on a horizontal line, i.e. synchronously, in the space-time diagram. Signal machines are studied in a series of articles named Abstract Geometrical Computation. In the present article, we design a signal machine that is able to synchronise/accumulate on any non-infinite slope. The slope is encoded in the initial configuration. This is done by constructing an infinite tree such that each node computes the way the tree expands. The interest of Abstract Geometrical computation is to do away with the constraint of discrete space, while tackling new difficulties from continuous space. The interest of this paper in particular is to provide basic tools for further study of computable accumulation lines in the signal machine model.
Iterative Network Pruning with Uncertainty Regularization for Lifelong Sentiment Classification
Venue: arXiv cs.CL 2021
Year: 2021

Lifelong learning capabilities are crucial for sentiment classifiers to process continuous streams of opinioned information on the Web. However, performing lifelong learning is non-trivial for deep neural networks as continually training of incrementally available information inevitably results in catastrophic forgetting or interference. In this paper, we propose a novel iterative network pruning with uncertainty regularization method for lifelong sentiment classification (IPRLS), which leverages the principles of network pruning and weight regularization. By performing network pruning with uncertainty regularization in an iterative manner, IPRLS can adapta single BERT model to work with continuously arriving data from multiple domains while avoiding catastrophic forgetting and interference. Specifically, we leverage an iterative pruning method to remove redundant parameters in large deep networks so that the freed-up space can then be employed to learn new tasks, tackling the catastrophic forgetting problem. Instead of keeping the old-tasks fixed when learning new tasks, we also use an uncertainty regularization based on the Bayesian online learning framework to constrain the update of old tasks weights in BERT, which enables positive backward transfer, i.e. learning new tasks improves performance on past tasks while protecting old knowledge from being lost. In addition, we propose a task-specific low-dimensional residual function in parallel to each layer of BERT, which makes IPRLS less prone to losing the knowledge saved in the base BERT network when learning a new task. Extensive experiments on 16 popular review corpora demonstrate that the proposed IPRLS method sig-nificantly outperforms the strong baselines for lifelong sentiment classification. For reproducibility, we submit the code and data at:https://github.com/siat-nlp/IPRLS.