OpenRaiser/Intern-Atlas huggingface dataset:
Attack-Centric Approach for Evaluating Transferability of Adversarial Samples in Machine Learning Models
Venue: arXiv cs.LG 2021
Year: 2021

Transferability of adversarial samples became a serious concern due to their impact on the reliability of machine learning system deployments, as they find their way into many critical applications. Knowing factors that influence transferability of adversarial samples can assist experts to make informed decisions on how to build robust and reliable machine learning systems. The goal of this study is to provide insights on the mechanisms behind the transferability of adversarial samples through an attack-centric approach. This attack-centric perspective interprets how adversarial samples would transfer by assessing the impact of machine learning attacks (that generated them) on a given input dataset. To achieve this goal, we generated adversarial samples using attacker models and transferred these samples to victim models. We analyzed the behavior of adversarial samples on victim models and outlined four factors that can influence the transferability of adversarial samples. Although these factors are not necessarily exhaustive, they provide useful insights to researchers and practitioners of machine learning systems.
A Survey: Deep Learning for Hyperspectral Image Classification with Few Labeled Samples
Venue: arXiv cs.CV 2021
Year: 2021

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.
Siamese BERT-based Model for Web Search Relevance Ranking Evaluated on a New Czech Dataset
Venue: arXiv cs.IR 2021
Year: 2021

Web search engines focus on serving highly relevant results within hundreds of milliseconds. Pre-trained language transformer models such as BERT are therefore hard to use in this scenario due to their high computational demands. We present our real-time approach to the document ranking problem leveraging a BERT-based siamese architecture. The model is already deployed in a commercial search engine and it improves production performance by more than 3%. For further research and evaluation, we release DaReCzech, a unique data set of 1.6 million Czech user query-document pairs with manually assigned relevance levels. We also release Small-E-Czech, an Electra-small language model pre-trained on a large Czech corpus. We believe this data will support endeavours both of search relevance and multilingual-focused research communities.
Episodic Policy Gradient Training
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a novel training procedure for policy gradient methods wherein episodic memory is used to optimize the hyperparameters of reinforcement learning algorithms on-the-fly. Unlike other hyperparameter searches, we formulate hyperparameter scheduling as a standard Markov Decision Process and use episodic memory to store the outcome of used hyperparameters and their training contexts. At any policy update step, the policy learner refers to the stored experiences, and adaptively reconfigures its learning algorithm with the new hyperparameters determined by the memory. This mechanism, dubbed as Episodic Policy Gradient Training (EPGT), enables an episodic learning process, and jointly learns the policy and the learning algorithm's hyperparameters within a single run. Experimental results on both continuous and discrete environments demonstrate the advantage of using the proposed method in boosting the performance of various policy gradient algorithms.
Fast Direct Stereo Visual SLAM
Venue: arXiv cs.RO 2021
Year: 2021

We propose a novel approach for fast and accurate stereo visual Simultaneous Localization and Mapping (SLAM) independent of feature detection and matching. We extend monocular Direct Sparse Odometry (DSO) to a stereo system by optimizing the scale of the 3D points to minimize photometric error for the stereo configuration, which yields a computationally efficient and robust method compared to conventional stereo matching. We further extend it to a full SLAM system with loop closure to reduce accumulated errors. With the assumption of forward camera motion, we imitate a LiDAR scan using the 3D points obtained from the visual odometry and adapt a LiDAR descriptor for place recognition to facilitate more efficient detection of loop closures. Afterward, we estimate the relative pose using direct alignment by minimizing the photometric error for potential loop closures. Optionally, further improvement over direct alignment is achieved by using the Iterative Closest Point (ICP) algorithm. Lastly, we optimize a pose graph to improve SLAM accuracy globally. By avoiding feature detection or matching in our SLAM system, we ensure high computational efficiency and robustness. Thorough experimental validations on public datasets demonstrate its effectiveness compared to the state-of-the-art approaches.
Generative Adversarial Networks for Synthetic Data Generation: A Comparative Study
Venue: arXiv cs.LG 2021
Year: 2021

Generative Adversarial Networks (GANs) are gaining increasing attention as a means for synthesising data. So far much of this work has been applied to use cases outside of the data confidentiality domain with a common application being the production of artificial images. Here we consider the potential application of GANs for the purpose of generating synthetic census microdata. We employ a battery of utility metrics and a disclosure risk metric (the Targeted Correct Attribution Probability) to compare the data produced by tabular GANs with those produced using orthodox data synthesis methods.
Introduction to Logical Entropy and its Relationship to Shannon Entropy
Venue: arXiv cs.IT 2021
Year: 2021

We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined." Logical entropy provides that definition. Logical entropy is the natural measure of the notion of information based on distinctions, differences, distinguishability, and diversity. It is the (normalized) quantitative measure of the distinctions of a partition on a set--just as the Boole-Laplace logical probability is the normalized quantitative measure of the elements of a subset of a set. And partitions and subsets are mathematically dual concepts--so the logic of partitions is dual in that sense to the usual Boolean logic of subsets, and hence the name "logical entropy." The logical entropy of a partition has a simple interpretation as the probability that a distinction or dit (elements in different blocks) is obtained in two independent draws from the underlying set. The Shannon entropy is shown to also be based on this notion of information-as-distinctions; it is the average minimum number of binary partitions (bits) that need to be joined to make all the same distinctions of the given partition. Hence all the concepts of simple, joint, conditional, and mutual logical entropy can be transformed into the corresponding concepts of Shannon entropy by a uniform non-linear dit-bit transform. And finally logical entropy linearizes naturally to the corresponding quantum concept. The quantum logical entropy of an observable applied to a state is the probability that two different eigenvalues are obtained in two independent projective measurements of that observable on that state. Keywords: logical entropy, Shannon entropy, partitions, MaxEntropy, quantum logical entropy, von Neumann entropy
Low-Resolution Massive MIMO Under Hardware Power Consumption Constraints
Venue: arXiv cs.IT 2021
Year: 2021

We consider a fully digital massive multiple-input multiple-output architecture with low-resolution analog-to-digital/digital-to-analog converters (ADCs/DACs) at the base station (BS) and analyze the performance trade-off between the number of BS antennas, the resolution of the ADCs/DACs, and the bandwidth. Assuming a hardware power consumption constraint, we determine the relationship between these design parameters by using a realistic model for the power consumption of the ADCs/DACs and the radio frequency chains. Considering uplink pilot-aided channel estimation, we build on the Bussgang decomposition to derive tractable expressions for uplink and downlink ergodic achievable sum rates. Numerical results show that the ergodic performance is boosted when many BS antennas with very low resolution (i.e., 2 to 3 bits) are adopted in both the uplink and the downlink.
Could AI Democratise Education? Socio-Technical Imaginaries of an EdTech Revolution
Venue: arXiv cs.CY 2021
Year: 2021

Artificial Intelligence (AI) in Education has been said to have the potential for building more personalised curricula, as well as democratising education worldwide and creating a Renaissance of new ways of teaching and learning. Millions of students are already starting to benefit from the use of these technologies, but millions more around the world are not. If this trend continues, the first delivery of AI in Education could be greater educational inequality, along with a global misallocation of educational resources motivated by the current technological determinism narrative. In this paper, we focus on speculating and posing questions around the future of AI in Education, with the aim of starting the pressing conversation that would set the right foundations for the new generation of education that is permeated by technology. This paper starts by synthesising how AI might change how we learn and teach, focusing specifically on the case of personalised learning companions, and then move to discuss some socio-technical features that will be crucial for avoiding the perils of these AI systems worldwide (and perhaps ensuring their success). This paper also discusses the potential of using AI together with free, participatory and democratic resources, such as Wikipedia, Open Educational Resources and open-source tools. We also emphasise the need for collectively designing human-centered, transparent, interactive and collaborative AI-based algorithms that empower and give complete agency to stakeholders, as well as support new emerging pedagogies. Finally, we ask what would it take for this educational revolution to provide egalitarian and empowering access to education, beyond any political, cultural, language, geographical and learning ability barriers.
I-WKNN: Fast-Speed and High-Accuracy WIFI Positioning for Intelligent Stadiums
Venue: arXiv cs.LG 2021
Year: 2021

Based on various existing wireless fingerprint location algorithms in intelligent sports venues, a high-precision and fast indoor location algorithm improved weighted k-nearest neighbor (I-WKNN) is proposed. In order to meet the complex environment of sports venues and the demand of high-speed sampling, this paper proposes an AP selection algorithm for offline and online stages. Based on the characteristics of the signal intensity distribution in intelligent venues, an asymmetric Gaussian filter algorithm is proposed. This paper introduces the application of the positioning algorithm in the intelligent stadium system, and completes the data acquisition and real-time positioning of the stadium. Compared with traditional WKNN and KNN algorithms, the I-WKNN algorithm has advantages in fingerprint positioning database processing, environmental noise adaptability, real-time positioning accuracy and positioning speed, etc. The experimental results show that the I-WKNN algorithm has obvious advantages in positioning accuracy and positioning time in a complex noise environment and has obvious application potential in a smart stadium.
Malakai: Music That Adapts to the Shape of Emotions
Venue: arXiv cs.MM 2021
Year: 2021

The advent of ML music models such as Google Magenta's MusicVAE now allow us to extract and replicate compositional features from otherwise complex datasets. These models allow computational composers to parameterize abstract variables such as style and mood. By leveraging these models and combining them with procedural algorithms from the last few decades, it is possible to create a dynamic song that composes music in real-time to accompany interactive experiences. Malakai is a tool that helps users of varying skill levels create, listen to, remix and share such dynamic songs. Using Malakai, a Composer can create a dynamic song that can be interacted with by a Listener
Hierarchical Optimal Transport for Unsupervised Domain Adaptation
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose a novel approach for unsupervised domain adaptation, that relates notions of optimal transport, learning probability measures and unsupervised learning. The proposed approach, HOT-DA, is based on a hierarchical formulation of optimal transport, that leverages beyond the geometrical information captured by the ground metric, richer structural information in the source and target domains. The additional information in the labeled source domain is formed instinctively by grouping samples into structures according to their class labels. While exploring hidden structures in the unlabeled target domain is reduced to the problem of learning probability measures through Wasserstein barycenter, which we prove to be equivalent to spectral clustering. Experiments on a toy dataset with controllable complexity and two challenging visual adaptation datasets show the superiority of the proposed approach over the state-of-the-art.
On the very accurate evaluation of the Voigt/complex error function with small imaginary argument
Venue: arXiv cs.MS 2021
Year: 2021

A rapidly convergent series, based on Taylor expansion of the imaginary part of the complex error function, is presented for highly accurate approximation of the Voigt/complex error function with small imaginary argument (Y less than 0.1). Error analysis and run-time tests in double-precision computing platform reveals that in the real and imaginary parts the proposed algorithm provides average accuracy exceeding 10^-15 and 10^-16, respectively, and the calculation speed is as fast as that of reported in recent publications. An optimized MATLAB code providing rapid computation with high accuracy is presented.
Improving the Reliability of Network Intrusion Detection Systems through Dataset Integration
Venue: arXiv cs.LG 2021
Year: 2021

This work presents Reliable-NIDS (R-NIDS), a novel methodology for Machine Learning (ML) based Network Intrusion Detection Systems (NIDSs) that allows ML models to work on integrated datasets, empowering the learning process with diverse information from different datasets. Therefore, R-NIDS targets the design of more robust models, that generalize better than traditional approaches. We also propose a new dataset, called UNK21. It is built from three of the most well-known network datasets (UGR'16, USNW-NB15 and NLS-KDD), each one gathered from its own network environment, with different features and classes, by using a data aggregation approach present in R-NIDS. Following R-NIDS, in this work we propose to build two well-known ML models (a linear and a non-linear one) based on the information of three of the most common datasets in the literature for NIDS evaluation, those integrated in UNK21. The results that the proposed methodology offers show how these two ML models trained as a NIDS solution could benefit from this approach, being able to generalize better when training on the newly proposed UNK21 dataset. Furthermore, these results are carefully analyzed with statistical tools that provide high confidence on our conclusions.
MIMO Networks with One-Bit ADCs: Receiver Design and Communication Strategies
Venue: arXiv cs.IT 2021
Year: 2021

High resolution analog to digital converters (ADCs) are conventionally used at the receiver terminals to store an accurate digital representation of the received signal, thereby allowing for reliable decoding of transmitted messages. However, in a wide range of applications, such as communication over millimeter wave and massive multiple-input multiple-output (MIMO) systems, the use of high resolution ADCs is not feasible due to power budget limitations. In the conventional fully digital receiver design, where each receiver antenna is connected to a distinct ADC, reducing the ADC resolution leads to performance loss in terms of achievable rates. One proposed method to mitigate the rate-loss is to use analog linear combiners leading to design of hybrid receivers. Here, the hybrid framework is augmented by the addition of delay elements to allow for temporal analog processing. Two new classes of receivers consisting of delay elements, analog linear combiners, and one-bit ADCs are proposed. The fundamental limits of communication in single and multi-user (uplink and downlink) MIMO systems employing the proposed receivers are investigated. In the high signal to noise ratio regime, it is shown that the proposed receivers achieve the maximum achievable rates among all receivers with the same number of one-bit ADCs.
Optimal Vehicle Path Planning Using Quadratic Optimization for Baidu Apollo Open Platform
Venue: arXiv cs.RO 2021
Year: 2021

Path planning is a key component in motion planning for autonomous vehicles. A path specifies the geometrical shape that the vehicle will travel, thus, it is critical to safe and comfortable vehicle motions. For urban driving scenarios, autonomous vehicles need the ability to navigate in cluttered environment, e.g., roads partially blocked by a number of vehicles/obstacles on the sides. How to generate a kinematically feasible and smooth path, that can avoid collision in complex environment, makes path planning a challenging problem. In this paper, we present a novel quadratic programming approach that generates optimal paths with resolution-complete collision avoidance capability.
Combining Embeddings and Fuzzy Time Series for High-Dimensional Time Series Forecasting in Internet of Energy Applications
Venue: arXiv cs.LG 2021
Year: 2021

The prediction of residential power usage is essential in assisting a smart grid to manage and preserve energy to ensure efficient use. An accurate energy forecasting at the customer level will reflect directly into efficiency improvements across the power grid system, however forecasting building energy use is a complex task due to many influencing factors, such as meteorological and occupancy patterns. In addiction, high-dimensional time series increasingly arise in the Internet of Energy (IoE), given the emergence of multi-sensor environments and the two way communication between energy consumers and the smart grid. Therefore, methods that are capable of computing high-dimensional time series are of great value in smart building and IoE applications. Fuzzy Time Series (FTS) models stand out as data-driven non-parametric models of easy implementation and high accuracy. Unfortunately, the existing FTS models can be unfeasible if all features were used to train the model. We present a new methodology for handling high-dimensional time series, by projecting the original high-dimensional data into a low dimensional embedding space and using multivariate FTS approach in this low dimensional representation. Combining these techniques enables a better representation of the complex content of multivariate time series and more accurate forecasts.
Optimizing Performance and Satisfaction in Matching and Movement Tasks in Virtual Reality with Interventions Using the Data Visualization Literacy Framework
Venue: arXiv cs.HC 2021
Year: 2021

Virtual reality (VR) has seen increased use for training and instruction. Designers can enable VR users to gain insights into their own performance by visualizing telemetry data from their actions in VR. Our ability to detect patterns and trends visually suggests the use of data visualization as a tool for users to identify strategies for improved performance. Typical tasks in VR training scenarios are manipulation of 3D objects (e.g., for learning how to maintain a jet engine) and navigation (e.g., to learn the geography of a building or landscape before traveling on-site). In this paper, we present the results of the RUI VR (84 subjects) and Luddy VR studies (68 subjects), where participants were divided into experiment and control cohorts. All subjects performed a series of tasks: 44 cube-matching tasks in RUI VR and 48 navigation tasks through a virtual building in Luddy VR (all divided into two sets). All Luddy VR subjects used VR gear; RUI VR subjects were divided across three setups: 2D Desktop (with laptop and mouse), VR Tabletop (in VR, sitting at a table), and VR Standup (in VR, standing). In an intervention called "Reflective phase," the experiment cohorts were presented with data visualizations, designed with the Data Visualization Literacy Framework (DVL-FW), of the data they generated during the first set of tasks before continuing to the second part of the study. For Luddy VR, we found that experiment users had significantly faster completion times in their second trial (p = 0.014) while scoring higher in a mid-questionnaire about the virtual building (p = 0.009). For RUI VR, we found no significant differences for completion time and accuracy between the two cohorts in the VR setups; however, 2D Desktop subjects in the experiment cohort had significantly higher rotation accuracy as well as satisfaction (p(rotation) = 0.031, p(satisfaction) = 0.040).
MAC address randomization tolerant crowd monitoring system using Wi-Fi packets
Venue: arXiv cs.NI 2021
Year: 2021

Media access control (MAC) addresses inside Wi-Fi packets can be used for beneficial activities such as crowdedness estimation, marketing, and hazard maps. However, the MAC address randomization systems introduced around 2014 make all conventional MAC-address-based crowd monitoring systems count the same device more than once. Therefore, there is a need to create a new crowd monitoring system tolerant to MAC address randomization to estimate the number of devices accurately. In this paper, Vision and TrueSight, two new crowd monitoring algorithms that estimate the number of devices, are proposed to prove that MAC-address-based crowd monitoring is still possible. In addition to probe requests, Vision uses data packets and beacon packets to mitigate the influence of randomization. Moreover, TrueSight uses sequence numbers and hierarchical clustering to estimate the number of devices. The experimental results of this study show that even without installing any special software, Vision can gather 440 randomly generated MAC addresses into one group and count only once, and TrueSight can estimate the number of devices with an accuracy of more than 75% with an acceptable error range of 1.