Reviewerly/exHarmony huggingface dataset:
Visible-Infrared Person Re-Identification Based on Frequency-Domain Simulated Multispectral Modality for Dual-Mode Cameras

With the prevalence of dual-mode cameras in surveillance systems, visible-infrared person re-identification (VI-ReID) has become an emerging topic. Existing studies of VI-ReID roughly fall into three categories: straightforwardly extracting features, improving loss functions, and conducting visible-infrared modality generation. The generation methods avoid the shortcoming of the former two that training models are generally vulnerable to parameter changes. However, these generation methods are usually based on spatial domain and are unavoidable to damage the original information of images. To tackle these limitations, we propose a novel frequency-domain simulated multispectral (FSMS) modality and visible-FSMS-infrared collaborative learning. FSMS modality consists of three-channel images generated by a channel-level reconstruction of visible images, primarily based on the nonsubsampled contourlet transform (NSCT) cooperating with a lightweight network. The generation exploits crucial spectral information and edge information contained in frequency domain. Then, we design a multi-modality network to conduct the tri-modality collaborative learning where FSMS modality is utilized as an intermediate, thereby preserving the original spatial structure of images. Additionally, a dynamic-weight tri-modality heterogeneous retrieval (THR) loss and a modality-shared classification (MSI) loss are devised to mine discriminative modality-invariant features. A cross-modality invariant (CMI) constraint for further exploring triplet-wise relationships and an intra-modality regularizer for relatively stable convergence are introduced. Finally, experimental results show that our algorithm significantly outperforms the latest state-of-the-arts by 5.7% and 4.4% CMC-1 accuracy on two mainstream benchmark datasets, respectively. And the reasons underlying the observed increase in performance are deeply discussed.
QMagFace: Simple and Accurate Quality-Aware Face Recognition

In this work, we propose QMagFace, a simple and effective face recognition solution (QMagFace) that combines a quality-aware comparison score with a recognition model based on a magnitude-aware angular margin loss. The proposed approach includes model-specific face image qualities in the comparison process to enhance the recognition performance under unconstrained circumstances. Exploiting the linearity between the qualities and their comparison scores induced by the utilized loss, our quality-aware comparison function is simple and highly generalizable. The experiments conducted on several face recognition databases and benchmarks demonstrate that the introduced quality-awareness leads to consistent improvements in the recognition performance. Moreover, the proposed QMagFace approach performs especially well under challenging circumstances, such as cross-pose, cross-age, or cross-quality. Consequently, it leads to state-of-the-art performances on several face recognition benchmarks, such as 98.50% on AgeDB, 83.95% on XQLFQ, and 98.74% on CFP-FP. The code for QMagFace is publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Towards Robust Object Detection in Floor Plan Images: A Data Augmentation Approach

Object detection is one of the most critical tasks in the field of Computer vision. This task comprises identifying and localizing an object in the image. Architectural floor plans represent the layout of buildings and apartments. The floor plans consist of walls, windows, stairs, and other furniture objects. While recognizing floor plan objects is straightforward for humans, automatically processing floor plans and recognizing objects is challenging. In this work, we investigate the performance of the recently introduced Cascade Mask R-CNN network to solve object detection in floor plan images. Furthermore, we experimentally establish that deformable convolution works better than conventional convolutions in the proposed framework. Prior datasets for object detection in floor plan images are either publicly unavailable or contain few samples. We introduce SFPI, a novel synthetic floor plan dataset consisting of 10,000 images to address this issue. Our proposed method conveniently exceeds the previous state-of-the-art results on the SESYD dataset with an mAP of 98.1%. Moreover, it sets impressive baseline results on our novel SFPI dataset with an mAP of 99.8%. We believe that introducing the modern dataset enables the researcher to enhance the research in this domain.
Generative high-capacity image hiding based on residual CNN in wavelet domain

Image hiding is the process of hiding a secret image in another meaningful image or other carriers so that the secret image remains imperceptible and can be recovered securely at the receiving end. The output image of an image hiding algorithm hides the secret image and visually appears to be the same as the carrier image, thus reducing the possibility of being attacked. The current hiding algorithms have relatively low hiding capacity and weak security. In this paper, we propose a generative image hiding algorithm based on a residual convolutional neural network (ResCNN) in wavelet domain to overcome the above-mentioned shortcomings. First, the secret image was subjected to wavelet transform. The low-frequency band of wavelet coefficients were discarded, and only the high-frequency bands were retained as features. These features were then effectively embedded into the carrier image by a generative ResCNN. The recovery network was trained simultaneously with the hiding network so as to extract the hidden features from the container and reconstruct the secret image. Pixel shuffle was used to recover a high-resolution secret image. The experimental results show that the proposed image hiding algorithm is capable of obtaining state-of-the-art results in terms of high hiding capacity and strong security measures.
An analysis of Single-Player Monte Carlo Tree Search performance in Sokoban

We apply the extension of Monte Carlo Tree Search for single player games (SP-MCTS) to Sokoban and compare its performance to a solver integrating Iterative Deepening A* (IDA*) with several problem-specific heuristics. We introduce two extensions of MCTS to deal with some of the challenges that Sokoban poses to MCTS methods, namely, the reduced search space that deadlock situations can cause and the large number of cycles. We also evaluate three domain-independent enhancements that have been shown to improve MCTS performance, namely, UCB1-Tuned, Rapid Action Value Estimation (RAVE), and Node Recycling. We perform a series of experiments to determine the best SP-MCTS configuration and then compare its performance to IDA*. We show that SP-MCTS can solve around 85% of the levels with 1000000 iterations, that is the same performance reached by IDA* with only 10000 nodes. Overall, our results suggest that IDA* is still the best solver for Sokoban, also because it can easily integrate much domain knowledge. At the same time, our results also highlight some interesting directions to design better MCTS solvers for this domain.
Contrastive Trajectory Learning for Tour Recommendation

The main objective of Personalized Tour Recommendation (PTR) is to generate a sequence of point-of-interest (POIs) for a particular tourist, according to the user-specific constraints such as duration time, start and end points, the number of attractions planned to visit, and so on. Previous PTR solutions are based on either heuristics for solving the orienteering problem to maximize a global reward with a specified budget or approaches attempting to learn user visiting preferences and transition patterns with the stochastic process or recurrent neural networks. However, existing learning methodologies rely on historical trips to train the model and use the next visited POI as the supervised signal, which may not fully capture the coherence of preferences and thus recommend similar trips to different users, primarily due to the data sparsity problem and long-tailed distribution of POI popularity. This work presents a novel tour recommendation model by distilling knowledge and supervision signals from the trips in a self-supervised manner. We propose Contrastive Trajectory Learning for Tour Recommendation (CTLTR), which utilizes the intrinsic POI dependencies and traveling intent to discover extra knowledge and augments the sparse data via pre-training auxiliary self-supervised objectives. CTLTR provides a principled way to characterize the inherent data correlations while tackling the implicit feedback and weak supervision problems by learning robust representations applicable for tour planning. We introduce a hierarchical recurrent encoder-decoder to identify tourists’ intentions and use the contrastive loss to discover subsequence semantics and their sequential patterns through maximizing the mutual information. Additionally, we observe that a data augmentation step as the preliminary of contrastive learning can solve the overfitting issue resulting from data sparsity. We conduct extensive experiments on a range of real-world datasets and demonstrate that our model can significantly improve the recommendation performance over the state-of-the-art baselines in terms of both recommendation accuracy and visiting orders.
Modeling Pointing for 3D Target Selection in VR

Virtual reality (VR) allows users to interact similarly to how they do in the physical world, such as touching, moving, and pointing at objects. To select objects at a distance, most VR techniques rely on casting a ray through one or two points located on the user's body (e.g., on the head and a finger), and placing a cursor on that ray. However, previous studies show that such rays do not help users achieve optimal pointing accuracy nor correspond to how they would naturally point. We seek to find features, which would best describe natural pointing at distant targets. We collect motion data from seven locations on the hand, arm, and body, while participants point at 27 targets across a virtual room. We evaluate the features of pointing and analyse sets of those for predicting pointing targets. Our analysis shows an 87% classification accuracy between the 27 targets for the best feature set and a mean distance of 23.56 cm in predicting pointing targets across the room. The feature sets can inform the design of more natural and effective VR pointing techniques for distant object selection.
Harnessing Artificial Intelligence for Health Message Generation: The Folic Acid Message Engine

Communication campaigns using social media can raise public awareness; however, they are difficult to sustain. A barrier is the need to generate and constantly post novel but on-topic messages, which creates a resource-intensive bottleneck.In this study, we aim to harness the latest advances in artificial intelligence (AI) to build a pilot system that can generate many candidate messages, which could be used for a campaign to suggest novel, on-topic candidate messages. The issue of folic acid, a B-vitamin that helps prevent major birth defects, serves as an example; however, the system can work with other issues that could benefit from higher levels of public awareness.We used the Generative Pretrained Transformer-2 architecture, a machine learning model trained on a large natural language corpus, and fine-tuned it using a data set of autodownloaded tweets about #folicacid. The fine-tuned model was then used as a message engine, that is, to create new messages about this topic. We conducted a web-based study to gauge how human raters evaluate AI-generated tweet messages compared with original, human-crafted messages.We found that the Folic Acid Message Engine can easily create several hundreds of new messages that appear natural to humans. Web-based raters evaluated the clarity and quality of a human-curated sample of AI-generated messages as on par with human-generated ones. Overall, these results showed that it is feasible to use such a message engine to suggest messages for web-based campaigns that focus on promoting awareness.The message engine can serve as a starting point for more sophisticated AI-guided message creation systems for health communication. Beyond the practical potential of such systems for campaigns in the age of social media, they also hold great scientific potential for the quantitative analysis of message characteristics that promote successful communication. We discuss future developments and obvious ethical challenges that need to be addressed as AI technologies for health persuasion enter the stage.
Efficient strategies for incremental mining of frequent closed itemsets over data streams

Mining frequent closed itemsets over data streams is an important data mining problem. Mining data streams is more challenging than mining static data because of the nature of data streams, including high arrival rate, massive volume of incoming data, and concept drift. The existing algorithms for mining frequent closed itemsets over data streams suffer from scalability and efficiency bottlenecks. This paper proposes a novel algorithm for mining frequent closed itemsets over data streams both for the sliding window model and for the landmark model. An indexed prefix closed itemset tree is proposed for compressing all closed itemsets and for quick searching of closed itemsets, and novel search strategies are proposed to prune the search space in updating the set of closed itemsets. The proposed algorithm outperforms the state-of-the-art intersection-based algorithms, CICLAD, ConPatSet, and CloStream, by several times to 2 orders of magnitude in efficiency, and also outperforms the state-of-the-art pattern enumeration algorithm, Moment, by up to 2 orders of magnitude over data streams with large windows and sparse data streams. The proposed algorithm is also superior in scalability.
Towards Measuring Fairness in Speech Recognition: Casual Conversations Dataset Transcriptions

The problem of machine learning systems demonstrating bias towards specific groups of individuals has been studied extensively, particularly in the Facial Recognition area, but much less so in Automatic Speech Recognition (ASR). This paper presents initial Speech Recognition results on “Casual Conversations” – a publicly released 846 hour corpus designed to help researchers evaluate their computer vision and audio models for accuracy across a diverse set of metadata, including age, gender, and skin tone. The entire corpus has been manually transcribed, allowing for detailed ASR evaluations across these metadata. Multiple ASR models are evaluated, including models trained on LibriSpeech, 14,000 hour transcribed, and over 2 million hour untranscribed social media videos. Significant differences in word error rate across gender and skin tone are observed at times for all models. We are releasing human transcripts from the Casual Conversations dataset to encourage the community to develop a variety of techniques to reduce these statistical biases.
IDEAL: An Open-Source Identifier Name Appraisal Tool

Developers must comprehend the code they will maintain, meaning that the code must be legible and reasonably self-descriptive. Unfortunately, there is still a lack of research and tooling that supports developers in understanding their naming practices; whether the names they choose make sense, whether they are consistent, and whether they convey the information required of them. In this paper, we present IDEAL, a tool that will provide feedback to developers about their identifier naming practices. Among its planned features, it will support linguistic anti-pattern detection, which is what will be discussed in this paper. IDEAL is designed to, and will, be extended to cover further anti-patterns, naming structures, and practices in the near future. IDEAL is open-source and publicly available, with a demo video available at: https://youtu.be/fVoOYGe50zg
BirdNet+: Two-Stage 3D Object Detection in LiDAR Through a Sparsity-Invariant Bird’s Eye View

Autonomous navigation relies upon an accurate understanding of the elements in the surroundings.Among the different on-board perception tasks, 3D object detection allows the identification of dynamic objects that cannot be registered by maps, being key for safe navigation.Thus, it often requires the use of LiDAR data, which is able to faithfully represent the scene geometry.However, although raw laser point clouds contain rich features to perform object detection, more compact representations such as the bird's eye view (BEV) projection are usually preferred in order to meet the time requirements of the control loop.This paper presents an end-to-end object detection network based on the well-known Faster R-CNN architecture that uses BEV images as input to produce the final 3D boxes.Our regression branches can infer not only the axis-aligned bounding boxes but also the rotation angle, height, and elevation of the objects in the scene.The proposed network provides state-of-the-art results for car, pedestrian, and cyclist detection with a single forward pass when evaluated on the KITTI 3D Object Detection Benchmark, with an accuracy that exceeds 64% mAP 3D for the Moderate difficulty.Further experiments on the challenging nuScenes dataset show the generalizability of both the method and the proposed BEV representation against different LiDAR devices and across a wider set of object categories by being able to reach more than 30% mAP with a single LiDAR sweep and almost 40% mAP with the usual 10-sweep accumulation.
Post-COVID-19 Mask-Aware Face Recognition System

Face recognition is a widely adopted technology in surveillance, border control, healthcare, banking services, and consumer product. Recent studies suggested that the performance of face recognition system is severely degraded in the presence of facial mask worn during COVID-19 pandemic. This work propose a mask-aware face recognition system that can identify subjects with and without facial mask presence. To this end, the two-fold contributions of this study are: (a) evaluation of the three hand-crafted descriptors, namely, Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG), and Local Directional Order Pattern (LDOP) along with Support Vector Machine (SVM) for facial mask detection, and (b) deep learning based mask-aware dynamic ensemble model that can recognize subjects in facial mask presence and absence. Experimental evaluation are conducted on the Real-World Masked Face Recognition Dataset (RMFRD) consisting of 426 subjects with 1945 and 88500 images with and without facial mask, respectively. Results suggest highest accuracy of 99.60% in facial mask detection using LDOP-based descriptor. The performance degradation of up to 24% was reported for deep learning based ResNet-50 face recognition model in the presence of facial mask. Performance due to our proposed dynamic ensemble model (in the presence and absence of mask) is at par with the performance of the baseline face recognition system in the absence of facial mask. For instance, Equal Error Rate (EER) of the proposed dynamic ensemble (mask-aware face recognition) in the presence of facial mask is 6.59% and that of the deep-learning based face recognition system is 6.42% in the absence of facial mask.
Image-based thickener mud layer height prediction with attention mechanism-based CNN

Mud layer height of thickener is the key quality index of thickening process which is difficult to achieve real-time detection with existing methods in reality. While the need of developing a soft sensor model which can be used for real-time detection of mud layer height, we proposed an end-to-end mud layer height prediction method with attention mechanism-based convolutional neural network (CNN). The dynamic features are firstly extracted from the image samples based on CNN, and then two types of attention mechanism are embedded sequentially to contribute to more precise prediction results. Compared with the traditional spatial attention mechanism, the regional spatial attention mechanism we proposed selectively divides the spatial feature map into regions, while regions containing important features are assigned larger weights. Adding the channel and regional spatial attention mechanism in CNN not only effectively improve both the precision and calculation speed, but also affect the dimension of the output feature map, so as to avoid the loss of channel or spatial attention information of the feature map. To verify the validity of the proposed method, different attention mechanisms are embedded in the CNN, and the corresponding experiments are carried out on the dataset of the thickener mud layer. The experimental results demonstrate the feasibility and effectiveness of the mud layer height prediction method.
DS-ADMM++: A Novel Distributed Quantized ADMM to Speed up Differentially Private Matrix Factorization

Matrix factorization is a powerful method to implement collaborative filtering recommender systems. This article addresses two major challenges, privacy and efficiency, which matrix factorization is facing. We based our work on DS-ADMM, a distributed matrix factorization algorithm with decent efficiency, to achieve the following two pieces of work: (1) Integrated local differential privacy paradigm into DS-ADMM to provide the privacy-preserving property; (2) Introduced a stochastic quantized function to reduce transmission overheads in ADMM to further improve efficiency. We named our work DS-ADMM++, in which one ’+’ refers to differential privacy, and the other ’+’ refers to quantized techniques. DS-ADMM++ is the first to perform efficient and private matrix factorization under the scenarios of differential privacy and DS-ADMM. We conducted experiments with benchmark data sets to demonstrate that our approach provides differential privacy and excellent scalability with a decent loss of accuracy.
It Takes Two to Techno-Tango: An Analysis of a Close Embrace Between Google/Apple and the EU in Fighting the Pandemic Through Contact Tracing Apps

All of us at Apple and Google believe there has never been a more important moment to work together to solve one of the world's most pressing problems. Through close cooperation and collaboration w...