Reviewerly/exHarmony huggingface dataset:
Seeing Through Darkness: Visual Localization at Night via Weakly Supervised Learning of Domain Invariant Features

Long term visual localization has to conquer the problem of matching images with dramatic photometric changes caused by different seasons, natural and man-made illumination changes, etc. Visual localization at night plays a vital role in many applications like autonomous driving and augmented reality, for which extracting keypoints and descriptors with robustness to day-night illumination changes has became the bottleneck. This paper proposes an adversarial learning based solution to harvest from the weakly domain labels of day and night images, along with the point level correspondences among day time images, to achieve robust local feature extraction and description across day-night images. The key idea is to learn a discriminator to distinguish whether a feature map is generated from the day or night images, and simultaneously to adjust the parameters of feature extraction network so as to fool the discriminator. After adversarial training of the discriminator and feature extraction network, the feature extraction network finally reaches a stable status so that the extracted feature maps are robust to day-night photometric changes, based on which day-night domain invariant keypoints and descriptors can be extracted. Compared to existing local feature learning methods, it only requires an additional set of easily captured night images to improve the domain invariance of learned features. Experiments on two challenging benchmarks show the effectiveness of proposed method. In addition, this paper revisits the widely used image matching metrics on HPatches and finds that recall of different methods is highly related to their relative localization performance.
Mixture Representation Learning for Deep Speaker Embedding

How to effectively convert a sequence of variable-length acoustic features to a fixed-dimension representation has always been a research focus in speaker recognition. In state-of-the-art speaker recognition systems, the conversion is implemented by concatenating the mean and the standard deviation of a sequence of frame-level features. However, a single mean and a single standard deviation are limited descriptive statistics for an acoustic sequence even with powerful feature extractors such as convolutional neural networks. In this paper, we propose a novel statistics pooling method that can produce more descriptive statistics through a mixture representation. Our approach is inspired by the expectation–maximization (EM) algorithm in Gaussian mixture models (GMMs). Instead of using traditional GMM style alignment, we novelly leverage modern deep learning tools to produce a more powerful mixture representation. The novelty includes: (1) unlike GMMs, the mixture assignments are determined by an attention network instead of the Euclidean distances between the frame-level features and explicit centers; (2) instead of using a single frame as input to the attention network, contextual frames are included to smooth out attention transition; and (3) soft-attention assignments are replaced by hard-attention assignments via the Gumbel-Softmax with straight-through estimators. With the proposed attention mechanism, we obtained a 13.7% relative improvement over vanilla mean and standard deviation pooling in the VOiCES19-eval set.
Probability Distribution on Full Rooted Trees

The recursive and hierarchical structure of full rooted trees is applicable to statistical models in various fields, such as data compression, image processing, and machine learning. In most of these cases, the full rooted tree is not a random variable; as such, model selection to avoid overfitting is problematic. One method to solve this problem is to assume a prior distribution on the full rooted trees. This enables the optimal model selection based on Bayes decision theory. For example, by assigning a low prior probability to a complex model, the maximum a posteriori estimator prevents the selection of the complex one. Furthermore, we can average all the models weighted by their posteriors. In this paper, we propose a probability distribution on a set of full rooted trees. Its parametric representation is suitable for calculating the properties of our distribution using recursive functions, such as the mode, expectation, and posterior distribution. Although such distributions have been proposed in previous studies, they are only applicable to specific applications. Therefore, we extract their mathematically essential components and derive new generalized methods to calculate the expectation, posterior distribution, etc.
Improving teaching in different disciplines of natural science and mathematics with innovative technologies

The study describes an innovative methodology for teaching natural and mathematical sciences in the context of distance learning using modern technological solutions and based on the concepts of active social learning that involves constructivist, problem-oriented, project and research approaches. The proposed methodology was tested on 80 students enrolled in two training courses in [BLINDED] University and [BLINDED] University, respectively: Mathematics Teaching in the Content and Language Integrated Learning (CLIL) context and Molecular Biology. The psychological safety of the proposed pedagogical methodology was investigated by assessing the level of psychological well-being of the participants in the educational process using the Scale of psychological well-being questionnaire developed by C. D. Ryff that was adapted to the educational context. The results of the study showed that the proposed pedagogical methodology was safe in relation to the life of students. It generally improved their perception of learning and themselves in the role of its participants, promoted positive self-esteem in group learning contacts, and, by developing learning involvement and interest through the Clil technology, qualitatively contributed to the achievement of educational progress by each student. The developed innovative methodology for teaching natural and mathematical sciences can be used as a pedagogical model for developing effective training courses. The demonstrated assessment of the level of psychological well-being adapted to the educational context can serve as a basis for the development of motivational learning strategies that support students in crisis learning conditions during the pandemic.
Architecture Disentanglement for Deep Neural Networks

Understanding the inner workings of deep neural networks (DNNs) is essential to provide trustworthy artificial intelligence techniques for practical applications. Existing studies typically involve linking semantic concepts to units or layers of DNNs, but fail to explain the inference process. In this paper, we introduce neural architecture disentanglement (NAD) to fill the gap. Specifically, NAD learns to disentangle a pre-trained DNN into sub-architectures according to independent tasks, forming information flows that describe the inference processes. We investigate whether, where, and how the disentanglement occurs through experiments conducted with handcrafted and automatically-searched network architectures, on both object-based and scene-based datasets. Based on the experimental results, we present three new findings that provide fresh insights into the inner logic of DNNs. First, DNNs can be divided into sub-architectures for independent tasks. Second, deeper layers do not always correspond to higher semantics. Third, the connection type in a DNN affects how the information flows across layers, leading to different disentanglement behaviors. With NAD, we further explain why DNNs sometimes give wrong predictions. Experimental results show that misclassified images have a high probability of being assigned to task sub-architectures similar to the correct ones. Our code is available at https://github.com/hujiecpp/NAD.
RGB camera-based fallen person detection system embedded on a mobile platform

Most injuries in the elderly are due to falls. The response time, to attend to the critically injured in such fall cases, is crucial to their survival. This paper presents a low-cost, autonomous assistive patrol robot which additionally includes a fallen person detection module with facial recognition that allows identification of patients. Patrol robots could be beneficial for care centers, where there is a considerable number of patients that require care. In these conditions, falls can be generally detected by the robotic platform during the post-fall phase. This allows the system to work with no frame rate constraints, allowing other tasks to be run simultaneously. Based on the YOLO network, we propose two approaches for the fallen person detector. The first approach can differentiate between fallen persons and persons doing ordinary activity in a single stage, while the second is a two-staged approach. The network weights were obtained using a fine-tuning process by retraining with our own extended Fall Person Dataset (E-FPDS), which we release as a benchmark for other RGB vision-based approaches. Quantitative evaluations confirm that the detector performs robustly in detecting fallen persons in different situations. The results also show a recall of 98.97% in our test set.
High quality proposal feature generation for crowded pedestrian detection

• Dual-region feature generation is proposed to eliminate false positives among different instances for crowded pedestrian detection. • The problem of pedestrian overlap is effectively dealt with by paired multi-instance prediction. • Paired non-maximum suppression is utilized to overcome the shortcomings of the original NMS and solve the problems of false and missed detection. Occlusion is a severe problem for pedestrian detection in crowded scenes. Due to the diversity of pedestrian postures and occlusion forms, leading to false detection and missed detection. In this paper, we propose a high quality proposal feature generation pedestrian detection algorithm to improve detection performance. Firstly, Dual-Region Feature Generation (DRFG) is proposed to generate high quality proposal features. Specifically, visible regions with less occlusion are introduced and low-precision proposals are generated for both the full-body and visible regions respectively. Then, proposals are respectively selected from the two kinds of proposals mentioned above to match in pairs, so as to guarantee a strong correspondence in information between the two proposals. Afterwards, the successfully matched proposal features are fused by Selective Kernel Feature Fusion (SKFF) to generate high quality proposal features. Secondly, Paired Multiple Instance Prediction(PMIP) is performed on the fused features to generate multiple prediction branches, and each prediction branch generates full-body and visible prediction box. Finally, Paired Non-Maximum Suppression(PNMS) is applied to the prediction boxes to reduce the false positives. Experiments have been conducted on CrowdHuman [1] and CityPersons [2] datasets. Comparing with baseline, our methods have achieved 5.9% A P and 1.5% M R − 2 improvement on the above two datasets, sufficiently verifying the effectiveness of our methods in crowded pedestrian detection.
An end-to-end deep learning system for requirements classification using recurrent neural networks

Existing requirements classification approaches mainly use lexical and syntactical features to classify requirements using both traditional machine learning and deep learning approaches with promising results. However, the existing techniques depend on word and sentence structures and employ preprocessing and feature engineering techniques to classify requirements from textual natural language documents. Moreover, existing studies deal with requirements classification as binary or multiclass classification problems and not as multilabel classification, although a given requirement can belong to multiple classes at the same time. The objective of this study is to classify requirements into functional and different non-functional types with minimal preprocessing and to model the task as a multilabel classification problem. In this paper, we use Bidirectional Gated Recurrent Neural Networks (BiGRU) to classify requirements using raw text. We investigated two different approaches: (i) using word sequences as tokens and (ii) using character sequences as tokens. Experiments conducted on the publicly available PROMISE and EHR datasets show the effectiveness of the presented techniques. We achieve state-of-the-art results on most of the tasks using word sequences as tokens. Requirements can be effectively classified into functional and different non-functional categories using the presented recurrent neural networks-based deep learning system, which involves minimal text prepossessing and no feature engineering.
Client Selection in Federated Learning under Imperfections in Environment

Federated learning promises an elegant solution for learning global models across distributed and privacy-protected datasets. However, challenges related to skewed data distribution, limited computational and communication resources, data poisoning, and free riding clients affect the performance of federated learning. Selection of the best clients for each round of learning is critical in alleviating these problems. We propose a novel sampling method named the irrelevance sampling technique. Our method is founded on defining a novel irrelevance score that incorporates the client characteristics in a single floating value, which can elegantly classify the client into three numerical sign defined pools for easy sampling. It is a computationally inexpensive, intuitive and privacy preserving sampling technique that selects a subset of clients based on quality and quantity of data on edge devices. It achieves 50–80% faster convergence even in highly skewed data distribution in the presence of free riders based on lack of data and severe class imbalance under both Independent and Identically Distributed (IID) and Non-IID conditions. It shows good performance on practical application datasets.
Learning-Based Channel Estimation and Phase Noise Compensation in Doubly-Selective Channels

In this letter, we propose a learning based channel estimation scheme for orthogonal frequency division multiplexing (OFDM) systems in the presence of phase noise in doubly-selective fading channels. Two-dimensional (2D) convolutional neural networks (CNNs) are employed for effective training and tracking of channel variation in both frequency as well as time domain. The proposed network learns and estimates the channel coefficients in the entire time-frequency (TF) grid based on pilots sparsely populated in the TF grid. In order to make the network robust to phase noise (PN) impairment, a novel training scheme where the training data is rotated by random phases before being fed to the network is employed. Further, using the estimated channel coefficients, a simple and effective PN estimation and compensation scheme is devised. Numerical results demonstrate that the proposed network and PN compensation scheme achieve robust OFDM performance in the presence of phase noise.
Design thinking for the everyday aestheticisation of urban renewable energy

Renewable energy infrastructures are becoming increasingly present in our environments, inevitably shaping the urban experiences of the everyday person as they move through the city. The profound impact these infrastructures have on social worlds has yet to be explored, with contemporary renewable energy discourse primarily focussing on the techno-economic. We argue for the everyday aestheticisation of renewable energy infrastructures, and how design thinking might offer a way forward in co-creating future meaningful experiences with renewable energy. We offer a collaborative design thinking workshop on the speculative experimentation of energy futures as a case study. The findings provide multi-scalar insights on exploring urban energy futures with citizen-designers – with aesthetics and lived experiences as central.
SLAMP: Stochastic Latent Appearance and Motion Prediction

Motion is an important cue for video prediction and often utilized by separating video content into static and dynamic components. Most of the previous work utilizing motion is deterministic but there are stochastic methods that can model the inherent uncertainty of the future. Existing stochastic models either do not reason about motion explicitly or make limiting assumptions about the static part. In this paper, we reason about appearance and motion in the video stochastically by predicting the future based on the motion history. Explicit reasoning about motion without history already reaches the performance of current stochastic models. The motion history further improves the results by allowing to predict consistent dynamics several frames into the future. Our model performs comparably to the state-of-the-art models on the generic video prediction datasets, however, significantly outperforms them on two challenging real-world autonomous driving datasets with complex motion and dynamic background.
Meaningful Secret Image Sharing with Saliency Detection

Secret image sharing (SIS), as one of the applications of information theory in information security protection, has been widely used in many areas, such as blockchain, identity authentication and distributed cloud storage. In traditional secret image sharing schemes, noise-like shadows introduce difficulties into shadow management and increase the risk of attacks. Meaningful secret image sharing is thus proposed to solve these problems. Previous meaningful SIS schemes have employed steganography to hide shares into cover images, and their covers are always binary images. These schemes usually include pixel expansion and low visual quality shadows. To improve the shadow quality, we design a meaningful secret image sharing scheme with saliency detection. Saliency detection is used to determine the salient regions of cover images. In our proposed scheme, we improve the quality of salient regions that are sensitive to the human vision system. In this way, we obtain meaningful shadows with better visual quality. Experiment results and comparisons demonstrate the effectiveness of our proposed scheme.
Data sharing in biological anthropology

Abstract Open data sharing democratizes science by making data more equitably available throughout the world. Furthermore, open data sharing improves the reproducibility and quality of research and enables new collaborations powered by the freely available data. Open data are defined as data that can be freely used, reused, and redistributed by anyone. For an interdisciplinary field like biological anthropology, data sharing is critical since one person cannot easily collect data across the domains relevant to our field. The goal of this paper is to encourage broader data sharing in our discipline by exploring the state of data sharing in the field of biological anthropology. Our paper is divided into four parts: the first section describes the benefits, challenges, and emerging solutions to open data sharing; the second section presents the results of our data archiving and sharing survey that was completed by over 700 researchers; the third section presents personal experiences of data sharing by the authors; and the fourth section discusses the strengths of different types of data repositories and provides a list of recommended data repositories.
Bernstein Polynomial-Based Method for Solving Optimal Trajectory Generation Problems

This paper presents a method for the generation of trajectories for autonomous system operations. The proposed method is based on the use of Bernstein polynomial approximations to transcribe infinite dimensional optimization problems into nonlinear programming problems. These, in turn, can be solved using off-the-shelf optimization solvers. The main motivation for this approach is that Bernstein polynomials possess favorable geometric properties and yield computationally efficient algorithms that enable a trajectory planner to efficiently evaluate and enforce constraints along the vehicles' trajectories, including maximum speed and angular rates as well as minimum distance between trajectories and between the vehicles and obstacles. By virtue of these properties and algorithms, feasibility and safety constraints typically imposed on autonomous vehicle operations can be enforced and guaranteed independently of the order of the polynomials. To support the use of the proposed method we introduce BeBOT (Bernstein/Bézier Optimal Trajectories), an open-source toolbox that implements the operations and algorithms for Bernstein polynomials. We show that BeBOT can be used to efficiently generate feasible and collision-free trajectories for single and multiple vehicles, and can be deployed for real-time safety critical applications in complex environments.
CaT: CAVS Traversability Dataset for Off-Road Autonomous Driving

In the context of autonomous driving, the existing semantic segmentation concept strongly supports on-road driving where hard inter-class boundaries are enforced and objects can be categorized based on their visible structures with high confidence. Due to the well-structured nature of typical on-road scenes, current road extraction processes are largely successful and most types of vehicles are able to traverse through the area that is detected as road. However, the off-road driving domain has many additional uncertainties such as uneven terrain structure, positive and negative obstacles, ditches, quagmires, hidden objects, etc. making it very unstructured. Traversing through such unstructured area is constrained by a vehicle’s type and its capability. Therefore, an alternative approach to segmentation of the off-road driving trail is required that supports consideration of the vehicle type in a way that is not considered in state-of-the-art on-road segmentation approaches. To overcome this limitation and facilitate the path extraction in the off-road driving domain, we propose traversability concept and corresponding dataset which is based on the notion that the driving trails should be finely resolved into different sub-trails and areas corresponding to the capability of different vehicle classes in order to achieve safe traversal. Based on this, we consider three different classes of vehicles (sedan, pickup, and off-road) and label the images corresponding to the traversing capability of those vehicles. So the proposed dataset facilitates the segmentation of off-road driving trail into three regions based on the nature of the driving area and vehicle capability. We call this dataset as CaT (CAVS Traversability, where CAVS stands for Center for Advanced Vehicular Systems) dataset and is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://www.cavs.msstate.edu/resources/downloads/CaT/CaT.tar.gz</uri> .
Augmented Lagrangian Adversarial Attacks

Adversarial attack algorithms are dominated by penalty methods, which are slow in practice, or more efficient distance-customized methods, which are heavily tailored to the properties of the distance considered. We propose a white-box attack algorithm to generate minimally perturbed adversarial examples based on Augmented Lagrangian principles. We bring several algorithmic modifications, which have a crucial effect on performance. Our attack enjoys the generality of penalty methods and the computational efficiency of distance-customized algorithms, and can be readily used for a wide set of distances. We compare our attack to state-of-the-art methods on three datasets and several models, and consistently obtain competitive performances with similar or lower computational complexity.