Reviewerly/exHarmony huggingface dataset:
Multi-interest Extraction Joint with Contrastive Learning for News Recommendation

News recommendation techniques aim to recommend candidate news to target user that he may be interested in, according to his browsed historical news. At present, existing works usually tend to represent user reading interest using a single vector during the modeling procedure. Actually, it is obviously that users usually have multiple and diverse interest in reality, such as sports, entertainment and so on. Thus it is irrational to represent user sophisticated semantic interest simply utilizing a single vector, which may conceal fine-grained information. In this work, we propose a novel method combining multi-interest extraction with contrastive learning, named MIECL, to tackle the above problem. Specifically, first, we construct several interest prototypes and design a multi-interest user encoder to learn multiple user representations under different interest conditions simultaneously. Then we adopt a graph-enhanced user encoder to enrich user corresponding semantic representation under each interest background through aggregating relevant information from neighbors. Finally, we contrast user multi-interest representations and interest prototype vectors to optimize the user representations themselves, in order to promote dissimilar semantic interest away from each other. We conduct experiments on two real-world news recommendation datasets MIND-Large, MIND-Small and empirical results demonstrate the effectiveness of our approach from multiple perspectives.
Powershap: A Power-Full Shapley Feature Selection Method

Abstract Feature selection is a crucial step in developing robust and powerful machine learning models. Feature selection techniques can be divided into two categories: filter and wrapper methods. While wrapper methods commonly result in strong predictive performances, they suffer from a large computational complexity and therefore take a significant amount of time to complete, especially when dealing with high-dimensional feature sets. Alternatively, filter methods are considerably faster, but suffer from several other disadvantages, such as (i) requiring a threshold value, (ii) many filter methods not taking into account intercorrelation between features, and (iii) ignoring feature interactions with the model. To this end, we present powershap , a novel wrapper feature selection method, which leverages statistical hypothesis testing and power calculations in combination with Shapley values for quick and intuitive feature selection. Powershap is built on the core assumption that an informative feature will have a larger impact on the prediction compared to a known random feature. Benchmarks and simulations show that powershap outperforms other filter methods with predictive performances on par with wrapper methods while being significantly faster, often even reaching half or a third of the execution time. As such, powershap provides a competitive and quick algorithm that can be used by various models in different domains. Furthermore, powershap is implemented as a plug-and-play and open-source sklearn component, enabling easy integration in conventional data science pipelines. User experience is even further enhanced by also providing an automatic mode that automatically tunes the hyper-parameters of the powershap algorithm, allowing to use the algorithm without any configuration needed.
Learning to Teach Fairness-Aware Deep Multi-task Learning

Fairness-aware learning mainly focuses on single task learning (STL). The fairness implications of multi-task learning (MTL) have only recently been considered and a seminal approach has been proposed that considers the fairness-accuracy trade-off for each task and the performance trade-off among different tasks. Instead of a rigid fairness-accuracy trade-off formulation, we propose a flexible approach that learns how to be fair in a MTL setting by selecting which objective (accuracy or fairness) to optimize at each step. We introduce the L2T-FMT algorithm that is a teacher-student network trained collaboratively; the student learns to solve the fair MTL problem while the teacher instructs the student to learn from either accuracy or fairness, depending on what is harder to learn for each task. Moreover, this dynamic selection of which objective to use at each step for each task reduces the number of trade-off weights from 2T to T, where T is the number of tasks. Our experiments on three real datasets show that L2T-FMT improves on both fairness (12–19%) and accuracy (up to 2%) over state-of-the-art approaches.
LSCALE: Latent Space Clustering-Based Active Learning for Node Classification

Node classification on graphs is an important task in many practical domains. It usually requires labels for training, which can be difficult or expensive to obtain in practice. Given a budget for labelling, active learning aims to improve performance by carefully choosing which nodes to label. Previous graph active learning methods learn representations using labelled nodes and select some unlabelled nodes for label acquisition. However, they do not fully utilize the representation power present in unlabelled nodes. We argue that the representation power in unlabelled nodes can be useful for active learning and for further improving performance of active learning for node classification. In this paper, we propose a latent space clustering-based active learning framework for node classification (LSCALE), where we fully utilize the representation power in both labelled and unlabelled nodes. Specifically, to select nodes for labelling, our framework uses the K-Medoids clustering algorithm on a latent space based on a dynamic combination of both unsupervised features and supervised features. In addition, we design an incremental clustering module to avoid redundancy between nodes selected at different steps. Extensive experiments on five datasets show that our proposed framework LSCALE consistently and significantly outperforms the state-of-the-art approaches by a large margin.
A Knowledge Infusion Based Multitasking System for Sarcasm Detection in Meme

In this paper, we hypothesize that sarcasm detection is closely associated with the emotion present in memes. Thereafter, we propose a deep multitask model to perform these two tasks in parallel, where sarcasm detection is treated as the primary task, and emotion recognition is considered an auxiliary task. We create a large-scale dataset consisting of 7416 memes in Hindi, one of the widely spoken languages. We collect the memes from various domains, such as politics, religious, racist, and sexist, and manually annotate each instance with three sarcasm categories, i.e., i) Not Sarcastic, ii) Mildly Sarcastic or iii) Highly Sarcastic and 13 fine-grained emotion classes. Furthermore, we propose a novel Knowledge Infusion (KI) based module which captures sentiment-aware representation from a pre-trained model using the Memotion dataset. Detailed empirical evaluation shows that the multitasking model performs better than the single-task model. We also show that using this KI module on top of our model can boost the performance of sarcasm detection in both single-task and multi-task settings even further. Code and dataset are available at this link: https://www. iitp.ac.in/ ai-nlp-ml/resources.html#Sarcastic-Meme-Detection .
In the studio with virtual reality: Student perceptions and potential integrations into art and design curriculum

The use of virtual reality (VR) to create immersive experiences through the principles taught in art and design curricula has been studied over the past two decades while, at the same time, demonstrating positive correlations between learning outcomes and the technology. Not surprisingly, the creation of 3D models within a virtual environment leads to a better understanding of 3D workflow. Yet, despite these developments, few studies have been conducted specifically on the application within a studio context for art and design disciplines and how VR might benefit artists and designers. This study confirms that the use of VR in art and design courses is perceived to promote a better understanding of three-dimensional spaces, design and to improve student-centred activities and student–student interactions. At the same time, integrating VR into curriculum should be strategic and aligned with learning outcomes for impactful learning experiences.
Accelerating Deep Neural Networks for Efficient Scene Understanding in Multi-Modal Automotive Applications

Environment perception constitutes one of the most critical operations performed by semi-and fully- autonomous vehicles. In recent years, Deep Neural Networks (DNNs) have become the standard tool for perception solutions owing to their impressive capabilities in analyzing and modelling complex and dynamic scenes, from (often muti-modal) sensory inputs. However, the well-established performance of DNNs comes at the cost of increased time and storage complexity, which may become problematic in automotive perception systems due to the requirement for a short prediction horizon (as in many cases inference must be performed in real time) and the limited computational, storage, and energy resources of mobile systems. A common way of addressing this problem is to transform the original large pre-trained networks into new smaller models, by utilizing Model Compression and Acceleration (MCA) techniques, improving both their storage and execution efficiency. Within the MCA framework, in this paper, we investigate the application of two state-of-the-art weight-sharing MCA techniques, namely a Vector Quantization (VQ) and a Dictionary Learning (DL) one, as well as two novel extensions, towards the acceleration and compression of widely used DNNs for 2D and 3D object-detection in automotive applications. Apart from the individual (uni-modal) networks, we also present and evaluate a multi-modal late-fusion algorithm for combining the detection results of the 2D and 3D detectors. Our evaluation studies are carried out on the KITTI Dataset. The obtained results lend themselves to a twofold interpretation. On the one hand, they showcase the significant acceleration and compression gains that can be achieved via the application of weight sharing on the selected DNN detectors, with limited accuracy loss, as well as highlight the performance differences between the two utilized weight sharing approaches. On the other, they demonstrate the substantial boost in detection performance obtained by combining the outcome of the two unimodal individual detectors, using the proposed late-fusion based multi-modal approach. Indeed, as our experiments have shown, pairing the high-performance DL-based MCA technique with the loss-mitigating effect of the multi-modal fusion approach, leads to highly accelerated models (up to approximately 2.5× and 6× for the 2D and 3D detectors, respectively) with the performance loss of the fused results ranging in most cases within single-digits figures (as low as around 1% for the class “cars”).
Quantization Robust Pruning With Knowledge Distillation

To resolve the problem that deep neural networks (DNN) require a large number of network parameters, many researchers have sought to compress the network. Network pruning, quantization and knowledge distillation have been studied for this purpose. Considering realistic scenarios such as deploying DNN on the resource constraint device where the network uploaded in the device performs wells in various bit-widths without re-training and the network with reasonable performance, we propose quantization robust pruning with knowledge distillation (QRPK) method. In QRPK, model weights are divided into essential weigths and inessential weights based on their magnitude value. Then, QRPK trains the quantization robustness model with a high pruning ratio by making the distribution of essential weights as a quantization friendly distribution. We conducted experiments on CIFAR-10 and CIFAR-100 to verify the effectiveness of QRPK and a QRPK trained model performs well in various bit-width, as it designed by pruning, quantization robustness and knowledge distillation.
The Prospects of Quantum Secure Communication for Secret Sharing

Quantum computing and secure communication have gained considerable interest in recent years, as quantum computing has a potent ability to accelerate certain complex problems, such as factoring problems and discrete logarithms, which are important foundations of traditional computer security. With the rapid development of quantum computers, some secure communication protocols based on certain mathematical problems will face a significant danger, and quantum properties can be used to construct quantum secure communication and quantum cryptography that can achieve unconditional security. Among many topics of quantum cryptography, quantum secret sharing (QSS) is an important and challenging issue that provides scenarios to store highly sensitive, important, and confidential information. In this article, we systematically illustrate the development of QSS and how it operates, as well as its challenging issues and future directions.
Driver Visual Attention Estimation Using Head Pose and Eye Appearance Information

In autonomous, as well as manually operated vehicles, monitoring the driver visual attention provides useful information about the behavior, intent and vigilance level of the driver. The gaze of the driver can be formulated in terms of a probabilistic visual map representing the region around which the driver’s attention is focused. The area of the estimated region changes based on the level of confidence of the estimation. This paper proposes a framework based on convolutional neural networks (CNNs) that takes the head pose and the eye appearance of the driver as inputs, and creates a fusion model that estimates the driver’s gaze on a 2D grid. The model contains upsampling layers to create estimations at multiple resolutions. The model is trained using data collected from 59 subjects with continuous recordings where the subject looks at a moving target in a parked car, and glances at a set of markers inside the car while driving the vehicle and while the car is parked. Our fusion framework provides superior performance than unimodal systems trained exclusively with head pose or eye appearance information. It estimates the gaze region with the target location lying within the 75% confidence region with an accuracy of 92.54%.
DroMOD: A Drone-Based Multi-Scope Object Detection System

Drone-based Multi-scope Object Detection (DroMOD) system targets an efficient detection of different kinds of objects using drones. DroMOD relies on a cross-platform framework where objects’ detection tasks are shared between the drone and the server. The drone stores a set of reference images about the supervised zone. Each time an image is captured with a well-defined spatial frequency, it is compared to its reference image and only ‘trigger images’ showing a change from the reference image are sent to the server. A Big Data streaming platform is deployed on server-side for scalable and efficient real-time object detection processing based on Deep Learning (DL) models. DroMOD system architecture allows for dynamically upgrading the DL models so that newly considered objects to detect can be added to the drone mission on the fly without modifying the drone embedded software. When compared to existing alternatives, DroMOD presents the best compromise between (i) object detection accuracy, (ii) real-time processing and (iii) resource efficiency. Since only lightweight processing is performed on the drone-side, memory and computation are highly optimised on drones. Furthermore, the drone-side image filtering is independent of the objects to detect and the object detection programs are deployed and updated only on the server-side which allows for multi-target detection with minimal engineering efforts and expertise.
The Omnirotor Platform: A Versatile, Multi-Modal, Coaxial, All-Terrain Vehicle

Mobile and aerial robots offer many potential applications, including warehouse logistics, surveillance, cinematography, and search and rescue.However, most such robots are task-specific and generally need more versatility to tackle multiple scenarios, terrains, and unstructured, dynamic environments.This paper presents the Omnirotor platform, a versatile, multi-modal, coaxial, tilt-rotor, all-terrain vehicle that combines an Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV) into a hybrid, all-terrain vehicle.The Omnirotor has two locomotion modes of operation (aerial and ground vehicle) and five operation configurations, as it can fly both in the normal and inverted configurations, and it can drive on the ground both in the normal and inverted configurations.It can also recover from any non-operational state to its normal, upside-down configuration.Moreover, in addition to the locomotion modes, the continuous omnidirectional thrust vectoring enables the Omnirotor platform to perform complex manipulation of objects.This work introduces the concept and discusses in detail the design, development, and experimental validation of the Omnirotor platform.In particular, it discusses the modeling and control schemes required by the different operation modes and configurations.It experimentally validates the platform's capabilities with experiments focusing on traversing challenging environments and unstructured, uneven terrains (e.g., a public park).Finally, the platform's ground, pushing-based manipulation capabilities are demonstrated through the execution of a puzzle-solving experiment where the solved puzzle serves as a landing platform for the all-terrain vehicle.The versatility of Omnirotor offers exciting prospects for use in challenging search-and-rescue scenarios, surveillance, and aerial and ground manipulation applications.
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition

The teacher-free online Knowledge Distillation (KD) aims to train an ensemble of multiple student models collaboratively and distill knowledge from each other. Although existing online KD methods achieve desirable performance, they often focus on class probabilities as the core knowledge type, ignoring the valuable feature representational information. We present a Mutual Contrastive Learning (MCL) framework for online KD. The core idea of MCL is to perform mutual interaction and transfer of contrastive distributions among a cohort of networks in an online manner. Our MCL can aggregate cross-network embedding information and maximize the lower bound to the mutual information between two networks. This enables each network to learn extra contrastive knowledge from others, leading to better feature representations, thus improving the performance of visual recognition tasks. Beyond the final layer, we extend MCL to intermediate layers and perform an adaptive layer-matching mechanism trained by meta-optimization. Experiments on image classification and transfer learning to visual recognition tasks show that layer-wise MCL can lead to consistent performance gains against state-of-the-art online KD approaches. The superiority demonstrates that layer-wise MCL can guide the network to generate better feature representations. Our code is publicly avaliable at https://github.com/winycg/L-MCL.
Modeling Semantic Correlation and Hierarchy for Real-World Wildlife Recognition

We explore the challenges of human-in-the-loop frameworks to label wildlife recognition datasets with a neural network. In wildlife imagery, the main challenges for a model to assist human annotation are two-fold: (1) the training dataset is usually imbalanced, which makes the model's suggestion biased, and (2) there are complex taxonomies in the classes. We establish a simple and efficient baseline, including the debiasing loss function and the hyperbolic network architecture, to address these issues. Moreover, we propose leveraging the semantic correlation to train the model more effectively by adding a co-occurrence layer to our model during training. We demonstrate the efficacy of our method in both a real-world wildlife areal survey recognition dataset and the public image classification dataset, CIFAR100-LT, CIFAR10-LT, and iNaturalist.
Class-Adaptive Data Augmentation for Image Classification

Data augmentation is a widely used regularization technique for improving the performance of convolutional neural networks (CNNs) in image classification tasks. To improve the effectiveness of data augmentation, it is important to find label-preserving transformations that fit the domain knowledge for a given dataset. In several real-world datasets, appropriate augmentation policies differ between classes, owing to their different characteristics. In this paper, we propose a class-adaptive data augmentation method that utilizes class-specific augmentation policies. First, we train the CNN without data augmentation. Subsequently, we derive a suitable augmentation policy for each class through an optimization procedure to maximize the degree of transformation while maintaining the label-preserving property of CNNs. Finally, we re-train the model using data augmentation based on derived class-specific augmentation policies. Through experiments using benchmark datasets with class-specific transformation constraints, we demonstrate that the proposed method achieves comparable or higher classification accuracy than the baseline methods using the same augmentation policy for all classes. Additionally, we confirm that the derived class-specific augmentation policies are consistent with the domain knowledge of each dataset.
Robust Cubature Kalman Filter With Gaussian-Multivariate Laplacian Mixture Distribution and Partial Variational Bayesian Method

This paper explores the problem of nonlinear state estimation in the presence of outlier-contaminated measurements. First, to deal with the non-stationary non-Gaussian noises caused by randomly occurring measurement outliers, we propose a new Gaussian-multivariate Laplacian mixture (GMLM) distribution and construct it as a hierarchical Gaussian expression. Next, utilizing the GMLM distribution and existing variational Bayesian (VB) method, a robust cubature Kalman filter is derived (VB-GMLMRCKF). Then, considering the high computational complexity of the existing VB inference process, a new partial VB (PVB) method is developed, which can separately estimate state vector and mismatched measurement noise covariance matrix. Building upon the VB-GMLMRCKF and PVB approach, a novel robust cubature Kalman filter is derived (PVB-GMLMRCKF). Finally, a target tracking model is utilized to evaluate the PVB-GMLMRCKF in terms of estimation accuracy, estimation consistency and computational efficiency.