OpenRaiser/Intern-Atlas huggingface dataset:
Using Affect as a Communication Modality to Improve Human-Robot Communication in Robot-Assisted Search and Rescue Scenarios
Venue: arXiv cs.RO 2022
Year: 2022

Emotions can provide a natural communication modality to complement the existing multi-modal capabilities of social robots, such as text and speech, in many domains. We conducted three online studies with 112, 223, and 151 participants to investigate the benefits of using emotions as a communication modality for Search And Rescue (SAR) robots. In the first experiment, we investigated the feasibility of conveying information related to SAR situations through robots' emotions, resulting in mappings from SAR situations to emotions. The second study used Affect Control Theory as an alternative method for deriving such mappings. This method is more flexible, e.g. allows for such mappings to be adjusted for different emotion sets and different robots. In the third experiment, we created affective expressions for an appearance-constrained outdoor field research robot using LEDs as an expressive channel. Using these affective expressions in a variety of simulated SAR situations, we evaluated the effect of these expressions on participants' (adopting the role of rescue workers) situational awareness. Our results and proposed methodologies provide (a) insights on how emotions could help conveying messages in the context of SAR, and (b) evidence on the effectiveness of adding emotions as a communication modality in a (simulated) SAR communication context.
HySAGE: A Hybrid Static and Adaptive Graph Embedding Network for Context-Drifting Recommendations
Venue: arXiv cs.IR 2022
Year: 2022

The recent popularity of edge devices and Artificial Intelligent of Things (AIoT) has driven a new wave of contextual recommendations, such as location based Point of Interest (PoI) recommendations and computing resource-aware mobile app recommendations. In many such recommendation scenarios, contexts are drifting over time. For example, in a mobile game recommendation, contextual features like locations, battery, and storage levels of mobile devices are frequently drifting over time. However, most existing graph-based collaborative filtering methods are designed under the assumption of static features. Therefore, they would require frequent retraining and/or yield graphical models burgeoning in sizes, impeding their suitability for context-drifting recommendations. In this work, we propose a specifically tailor-made Hybrid Static and Adaptive Graph Embedding (HySAGE) network for context-drifting recommendations. Our key idea is to disentangle the relatively static user-item interaction and rapidly drifting contextual features. Specifically, our proposed HySAGE network learns a relatively static graph embedding from user-item interaction and an adaptive embedding from drifting contextual features. These embeddings are incorporated into an interest network to generate the user interest in some certain context. We adopt an interactive attention module to learn the interactions among static graph embeddings, adaptive contextual embeddings, and user interest, helping to achieve a better final representation. Extensive experiments on real-world datasets demonstrate that HySAGE significantly improves the performance of the existing state-of-the-art recommendation algorithms.
Looking For A Match: Self-supervised Clustering For Automatic Doubt Matching In e-learning Platforms
Venue: arXiv cs.LG 2022
Year: 2022

Recently, e-learning platforms have grown as a place where students can post doubts (as a snap taken with smart phones) and get them resolved in minutes. However, the significant increase in the number of student-posted doubts with high variance in quality on these platforms not only presents challenges for teachers' navigation to address them but also increases the resolution time per doubt. Both are not acceptable, as high doubt resolution time hinders the students learning progress. This necessitates ways to automatically identify if there exists a similar doubt in repository and then serve it to the teacher as the plausible solution to validate and communicate with the student. Supervised learning techniques (like Siamese architecture) require labels to identify the matches, which is not feasible as labels are scarce and expensive. In this work, we, thus, developed a label-agnostic doubt matching paradigm based on the representations learnt via self-supervised technique. Building on prior theoretical insights of BYOL (bootstrap your own latent space), we propose custom BYOL which combines domain-specific augmentation with contrastive objective over a varied set of appropriately constructed data views. Results highlighted that, custom BYOL improves the top-1 matching accuracy by approximately 6\% and 5\% as compared to both BYOL and supervised learning instances, respectively. We further show that both BYOL-based learning instances performs either on par or better than human labeling.
Weighted Maximum Entropy Inverse Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

We study inverse reinforcement learning (IRL) and imitation learning (IM), the problems of recovering a reward or policy function from expert's demonstrated trajectories. We propose a new way to improve the learning process by adding a weight function to the maximum entropy framework, with the motivation of having the ability to learn and recover the stochasticity (or the bounded rationality) of the expert policy. Our framework and algorithms allow to learn both a reward (or policy) function and the structure of the entropy terms added to the Markov Decision Processes, thus enhancing the learning procedure. Our numerical experiments using human and simulated demonstrations and with discrete and continuous IRL/IM tasks show that our approach outperforms prior algorithms.
Pretrained Language Encoders are Natural Tagging Frameworks for Aspect Sentiment Triplet Extraction
Venue: arXiv cs.CL 2022
Year: 2022

Aspect Sentiment Triplet Extraction (ASTE) aims to extract the spans of aspect, opinion, and their sentiment relations as sentiment triplets. Existing works usually formulate the span detection as a 1D token tagging problem, and model the sentiment recognition with a 2D tagging matrix of token pairs. Moreover, by leveraging the token representation of Pretrained Language Encoders (PLEs) like BERT, they can achieve better performance. However, they simply leverage PLEs as feature extractors to build their modules but never have a deep look at what specific knowledge does PLEs contain. In this paper, we argue that instead of further designing modules to capture the inductive bias of ASTE, PLEs themselves contain "enough" features for 1D and 2D tagging: (1) The token representation contains the contextualized meaning of token itself, so this level feature carries necessary information for 1D tagging. (2) The attention matrix of different PLE layers can further capture multi-level linguistic knowledge existing in token pairs, which benefits 2D tagging. (3) Furthermore, with simple transformations, these two features can also be easily converted to the 2D tagging matrix and 1D tagging sequence, respectively. That will further boost the tagging results. By doing so, PLEs can be natural tagging frameworks and achieve a new state of the art, which is verified by extensive experiments and deep analyses.
Fully Automated End-to-End Fake Audio Detection
Venue: arXiv cs.SD 2022
Year: 2022

The existing fake audio detection systems often rely on expert experience to design the acoustic features or manually design the hyperparameters of the network structure. However, artificial adjustment of the parameters can have a relatively obvious influence on the results. It is almost impossible to manually set the best set of parameters. Therefore this paper proposes a fully automated end-toend fake audio detection method. We first use wav2vec pre-trained model to obtain a high-level representation of the speech. Furthermore, for the network structure, we use a modified version of the differentiable architecture search (DARTS) named light-DARTS. It learns deep speech representations while automatically learning and optimizing complex neural structures consisting of convolutional operations and residual blocks. The experimental results on the ASVspoof 2019 LA dataset show that our proposed system achieves an equal error rate (EER) of 1.08%, which outperforms the state-of-the-art single system.
Graph analytics workflows enactment on just in time data centres, Position Paper
Venue: arXiv cs.DB 2022
Year: 2022

This paper discusses our vision of multirole-capable decision-making systems across a broad range of Data Science (DS) workflows working on graphs through disaggregated data centres. Our vision is that an alternative is possible to work on a disaggregated solution for the provision of computational services under the notion of a disaggregated data centre. We define this alternative as a virtual entity that dynamically provides resources crosscutting the layers of edge, fog and data centre according to the workloads submitted by the workflows and their Service Level Objectives.
Finding Emotions in Faces: A Meta-Classifier
Venue: arXiv cs.CV 2022
Year: 2022

Machine learning has been used to recognize emotions in faces, typically by looking for 8 different emotional states (neutral, happy, sad, surprise, fear, disgust, anger and contempt). We consider two approaches: feature recognition based on facial landmarks and deep learning on all pixels; each produced 58% overall accuracy. However, they produced different results on different images and thus we propose a new meta-classifier combining these approaches. It produces far better results with 77% accuracy
SnowFormer: Context Interaction Transformer with Scale-awareness for Single Image Desnowing
Venue: arXiv cs.CV 2022
Year: 2022

Due to various and complicated snow degradations, single image desnowing is a challenging image restoration task. As prior arts can not handle it ideally, we propose a novel transformer, SnowFormer, which explores efficient cross-attentions to build local-global context interaction across patches and surpasses existing works that employ local operators or vanilla transformers. Compared to prior desnowing methods and universal image restoration methods, SnowFormer has several benefits. Firstly, unlike the multi-head self-attention in recent image restoration Vision Transformers, SnowFormer incorporates the multi-head cross-attention mechanism to perform local-global context interaction between scale-aware snow queries and local-patch embeddings. Second, the snow queries in SnowFormer are generated by the query generator from aggregated scale-aware features, which are rich in potential clean cues, leading to superior restoration results. Third, SnowFormer outshines advanced state-of-the-art desnowing networks and the prevalent universal image restoration transformers on six synthetic and real-world datasets. The code is released in \url{https://github.com/Ephemeral182/SnowFormer}.
C$^{2}$IMUFS: Complementary and Consensus Learning-based Incomplete Multi-view Unsupervised Feature Selection
Venue: arXiv cs.LG 2022
Year: 2022

Multi-view unsupervised feature selection (MUFS) has been demonstrated as an effective technique to reduce the dimensionality of multi-view unlabeled data. The existing methods assume that all of views are complete. However, multi-view data are usually incomplete, i.e., a part of instances are presented on some views but not all views. Besides, learning the complete similarity graph, as an important promising technology in existing MUFS methods, cannot achieve due to the missing views. In this paper, we propose a complementary and consensus learning-based incomplete multi-view unsupervised feature selection method (C$^{2}$IMUFS) to address the aforementioned issues. Concretely, C$^{2}$IMUFS integrates feature selection into an extended weighted non-negative matrix factorization model equipped with adaptive learning of view-weights and a sparse $\ell_{2,p}$-norm, which can offer better adaptability and flexibility. By the sparse linear combinations of multiple similarity matrices derived from different views, a complementary learning-guided similarity matrix reconstruction model is presented to obtain the complete similarity graph in each view. Furthermore, C$^{2}$IMUFS learns a consensus clustering indicator matrix across different views and embeds it into a spectral graph term to preserve the local geometric structure. Comprehensive experimental results on real-world datasets demonstrate the effectiveness of C$^{2}$IMUFS compared with state-of-the-art methods.
Strategic Communication with Cost-Dependent Decoders via the Gray-Wyner Network
Venue: arXiv cs.IT 2022
Year: 2022

In decentralized decision-making problems, communicating agents choose their actions based on locally available information and knowledge about decision rules or strategies of other agents. In this work, we consider a strategic communication game between an informed encoder and two decoders communicating via a Gray-Wyner network. All three agents are assumed to be rational and endowed with distinct objectives captured by non-aligned cost functions. The encoder selects and announces beforehand the compression scheme to be implemented. Then, it transmits three signals: a public signal, and a private signal to each decoder inducing a Bayesian game among the decoders. Our goal is to characterize the encoder's minimal long-run cost function subject to the optimal compression scheme that satisfies both decoders incentives constraints.
JVLDLoc: a Joint Optimization of Visual-LiDAR Constraints and Direction Priors for Localization in Driving Scenario
Venue: arXiv cs.CV 2022
Year: 2022

The ability for a moving agent to localize itself in environment is the basic demand for emerging applications, such as autonomous driving, etc. Many existing methods based on multiple sensors still suffer from drift. We propose a scheme that fuses map prior and vanishing points from images, which can establish an energy term that is only constrained on rotation, called the direction projection error. Then we embed these direction priors into a visual-LiDAR SLAM system that integrates camera and LiDAR measurements in a tightly-coupled way at backend. Specifically, our method generates visual reprojection error and point to Implicit Moving Least Square(IMLS) surface of scan constraints, and solves them jointly along with direction projection error at global optimization. Experiments on KITTI, KITTI-360 and Oxford Radar Robotcar show that we achieve lower localization error or Absolute Pose Error (APE) than prior map, which validates our method is effective.
RGBD1K: A Large-scale Dataset and Benchmark for RGB-D Object Tracking
Venue: arXiv cs.CV 2022
Year: 2022

RGB-D object tracking has attracted considerable attention recently, achieving promising performance thanks to the symbiosis between visual and depth channels. However, given a limited amount of annotated RGB-D tracking data, most state-of-the-art RGB-D trackers are simple extensions of high-performance RGB-only trackers, without fully exploiting the underlying potential of the depth channel in the offline training stage. To address the dataset deficiency issue, a new RGB-D dataset named RGBD1K is released in this paper. The RGBD1K contains 1,050 sequences with about 2.5M frames in total. To demonstrate the benefits of training on a larger RGB-D data set in general, and RGBD1K in particular, we develop a transformer-based RGB-D tracker, named SPT, as a baseline for future visual object tracking studies using the new dataset. The results, of extensive experiments using the SPT tracker emonstrate the potential of the RGBD1K dataset to improve the performance of RGB-D tracking, inspiring future developments of effective tracker designs. The dataset and codes will be available on the project homepage: https://github.com/xuefeng-zhu5/RGBD1K.
Stop&Hop: Early Classification of Irregular Time Series
Venue: arXiv cs.LG 2022
Year: 2022

Early classification algorithms help users react faster to their machine learning model's predictions. Early warning systems in hospitals, for example, let clinicians improve their patients' outcomes by accurately predicting infections. While early classification systems are advancing rapidly, a major gap remains: existing systems do not consider irregular time series, which have uneven and often-long gaps between their observations. Such series are notoriously pervasive in impactful domains like healthcare. We bridge this gap and study early classification of irregular time series, a new setting for early classifiers that opens doors to more real-world problems. Our solution, Stop&Hop, uses a continuous-time recurrent network to model ongoing irregular time series in real time, while an irregularity-aware halting policy, trained with reinforcement learning, predicts when to stop and classify the streaming series. By taking real-valued step sizes, the halting policy flexibly decides exactly when to stop ongoing series in real time. This way, Stop&Hop seamlessly integrates information contained in the timing of observations, a new and vital source for early classification in this setting, with the time series values to provide early classifications for irregular time series. Using four synthetic and three real-world datasets, we demonstrate that Stop&Hop consistently makes earlier and more-accurate predictions than state-of-the-art alternatives adapted to this new problem. Our code is publicly available at https://github.com/thartvigsen/StopAndHop.
Rate-Splitting Multiple Access for Intelligent Reflecting Surface-Aided Secure Transmission
Venue: arXiv cs.IT 2022
Year: 2022

In this letter, we study a rate-splitting multiple access (RSMA)-based intelligent reflecting surface (IRS)-aided multi-user multiple-input single-output (MISO) secure communication system with a potential eavesdropper (Eve). Aiming to maximize the minimum secrecy rate (SR) among all the legitimate users (LUs), a design problem for jointly optimizing the transmit beamforming with artificial noise (AN), the IRS beamforming, and the secrecy common rate allocation is formulated. Since the design problem is highly non-convex with coupled optimization variables, we develop a computationally efficient algorithm based on the alternating optimization (AO) technique to solve it suboptimally. Numerical results demonstrate that the proposed design can significantly improve the max-min SR over the benchmark schemes without IRS or adopting other multiple access techniques. In particular, employing the RSMA strategy can substantially reduce the required numbers of IRS elements for achieving a target level of secrecy performance compared with the benchmark schemes.
On The Robustness of Channel Allocation in Joint Radar And Communication Systems: An Auction Approach
Venue: arXiv cs.GT 2022
Year: 2022

Joint radar and communication (JRC) is a promising technique for spectrum re-utilization, which enables radar sensing and data transmission to operate on the same frequencies and the same devices. However, due to the multi-objective property of JRC systems, channel allocation to JRC nodes should be carefully designed to maximize system performance. Additionally, because of the broadcast nature of wireless signals, a watchful adversary, i.e., a warden, can detect ongoing transmissions and attack the system. Thus, we develop a covert JRC system that minimizes the detection probability by wardens, in which friendly jammers are deployed to improve the covertness of the JRC nodes during radar sensing and data transmission operations. Furthermore, we propose a robust multi-item auction design for channel allocation for such a JRC system that considers the uncertainty in bids. The proposed auction mechanism achieves the properties of truthfulness, individual rationality, budget feasibility, and computational efficiency. The simulations clearly show the benefits of our design to support covert JRC systems and to provide incentive to the JRC nodes in obtaining spectrum, in which the auction-based channel allocation mechanism is robust against perturbations in the bids, which is highly effective for JRC nodes working in uncertain environments.
qDWI-Morph: Motion-compensated quantitative Diffusion-Weighted MRI analysis for fetal lung maturity assessment
Venue: arXiv cs.CV 2022
Year: 2022

Quantitative analysis of fetal lung Diffusion-Weighted MRI (DWI) data shows potential in providing quantitative imaging biomarkers that indirectly reflect fetal lung maturation. However, fetal motion during the acquisition hampered quantitative analysis of the acquired DWI data and, consequently, reliable clinical utilization. We introduce qDWI-morph, an unsupervised deep-neural-network architecture for motion compensated quantitative DWI (qDWI) analysis. Our approach couples a registration sub-network with a quantitative DWI model fitting sub-network. We simultaneously estimate the qDWI parameters and the motion model by minimizing a bio-physically-informed loss function integrating a registration loss and a model fitting quality loss. We demonstrated the added-value of qDWI-morph over: 1) a baseline qDWI analysis without motion compensation and 2) a baseline deep-learning model incorporating registration loss solely. The qDWI-morph achieved a substantially improved correlation with the gestational age through in-vivo qDWI analysis of fetal lung DWI data (R-squared=0.32 vs. 0.13, 0.28). Our qDWI-morph has the potential to enable motion-compensated quantitative analysis of DWI data and to provide clinically feasible bio-markers for non-invasive fetal lung maturity assessment. Our code is available at: https://github.com/TechnionComputationalMRILab/qDWI-Morph.
Comparison-based Conversational Recommender System with Relative Bandit Feedback
Venue: arXiv cs.IR 2022
Year: 2022

With the recent advances of conversational recommendations, the recommender system is able to actively and dynamically elicit user preference via conversational interactions. To achieve this, the system periodically queries users' preference on attributes and collects their feedback. However, most existing conversational recommender systems only enable the user to provide absolute feedback to the attributes. In practice, the absolute feedback is usually limited, as the users tend to provide biased feedback when expressing the preference. Instead, the user is often more inclined to express comparative preferences, since user preferences are inherently relative. To enable users to provide comparative preferences during conversational interactions, we propose a novel comparison-based conversational recommender system. The relative feedback, though more practical, is not easy to be incorporated since its feedback scale is always mismatched with users' absolute preferences. With effectively collecting and understanding the relative feedback from an interactive manner, we further propose a new bandit algorithm, which we call RelativeConUCB. The experiments on both synthetic and real-world datasets validate the advantage of our proposed method, compared to the existing bandit algorithms in the conversational recommender systems.