pwc-archive/papers-with-abstracts huggingface dataset:
Dense Interspecies Face Embedding
Authors: Sejong Yang, Subin Jeon, Seonghyeon Nam, Seon Joo Kim
Date: 2022-11-28 00:00:00
Proceeding: NeruIPS 2022 11
Tasks: Image Manipulation, Interspecies Facial Keypoint Transfer, Keypoint Detection, Knowledge Distillation

Dense Interspecies Face Embedding (DIFE) is a new direction for understanding faces of various animals by extracting common features among animal faces including human face. There are three main obstacles for interspecies face understanding: (1) lack of animal data compared to human, (2) ambiguous connection between faces of various animals, and (3) extreme shape and style variance. To cope with the lack of data, we utilize multi-teacher knowledge distillation of CSE and StyleGAN2 requiring no additional data or label. Then we synthesize pseudo pair images through the latent space exploration of StyleGAN2 to find implicit associations between different animal faces. Finally, we introduce the semantic matching loss to overcome the problem of extreme shape differences between species. To quantitatively evaluate our method over possible previous methodologies like unsupervised keypoint detection, we perform interspecies facial keypoint transfer on MAFL and AP-10K. Furthermore, the results of other applications like interspecies face image manipulation and dense keypoint transfer are provided. The code is available at https://github.com/kingsj0405/dife.
Passive-Aggressive online learning with nonlinear embeddings
Authors: Javier Jorge, Roberto Paredes
Date: 2018-01-31 00:00:00
Proceeding: Pattern Recognition 2018 1
Tasks: Binary Classification

Nowadays, there is an increasing demand for machine learning techniques which can deal with problems where the instances are produced as a stream or in real time. In these scenarios, online learning is able to learn a model from data that comes continuously. The adaptability, efficiency and scalability of online learning techniques have been gaining interest last years with the increasing amount of data generated every day. In this paper, we propose a novel binary classification approach based on nonlinear mapping functions under an online learning framework. The non-convex optimization problem that arises is split into three different convex problems that are solved by means of Passive-Aggressive Online Learning. We evaluate both the adaptability and generalization of our model through several experiments comparing with the state of the art techniques. We improve significantly the results in several datasets widely used previously by the online learning community.
DMML-Net: Deep Metametric Learning for Few-Shot Geographic Object Segmentation in Remote Sensing Imagery
Authors: Bing Wang, Zhirui Wang, Xian Sun, Hongqi Wang, Kun fu
Date: 2021-10-08 00:00:00
Proceeding: IEEE Transactions on Geoscience and Remote Sensing 2021 10
Tasks: Metric Learning, Segmentation, Semantic Segmentation

Geographic object segmentation is a fundamental yet challenging problem for remote sensing image interpretation. The prevalent paradigm to solve this problem is to train deep neural networks on massive labeled samples. Although remarkable achievements have been attained, these methods suffer from the severe dependence on the large-scale dataset and require a long training process with high computation burden. To address these issues, a deep metametric learning framework, named DMML-Net, consisting of the metametric learner and the base-metric learner, is proposed for few-shot geographic object segmentation. First, DMML-Net formulates the segmentation as the metric-based pixel classification and develops a deep feature pyramid comparison network as the architecture of the metric learner for multiscale metric learning. Benefiting from this design, the segmentation can be efficiently solved, as well as being robust to deal with the scale variations of geographic objects. Second, an affinity-based fusion mechanism is introduced to adaptively reweight and fuse the semantic information across samples, effectively calibrating the deviation of prototypes induced by the intraclass variations. Third, considering the impact of the large interclass distribution divergences, DMML-Net presents a metametric training paradigm to provide the metric model with flexible scalability for fast adaptation to novel tasks. After metatraining, DMML-Net can be applied for the few-shot segmentation tasks of novel geographic objects with only a few gradient steps on the small training set. Experimental results on two benchmark remote sensing datasets demonstrate the validity and the superiority of our method in low-shot conditions where there are only one to ten labeled samples.
Emphasizing Complementary Samples for Non-literal Cross-modal Retrieval
Authors: Christopher Thomas, Adriana Kovashka
Date: 2022-06-25 00:00:00
Proceeding: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2022 6
Tasks: Cross-Modal Retrieval, Retrieval, Semantic Similarity, Semantic Textual Similarity

Existing cross-modal retrieval methods assume a straightforward relationship where images and text contain portrayals or mentions of the same objects. In contrast, real-world image-text pairs (e.g. an image and its caption in a news article) often feature more complex relations. Importantly, not all image-text pairs have the same relationship: in some pairs, image and text may be more closely aligned, while others are more loosely aligned hence complementary. In order to ensure the model learns a semantically robust space which captures nuanced relationships, care must be taken that loosely-aligned image-text pairs have a strong enough impact on learning. In this paper, we propose a novel approach to prioritize loosely-aligned samples. Unlike prior sample weighting methods, ours relies on estimating to what extent semantic similarity is preserved in the separate channels (images/text) in the learned multimodal space. In particular, the image-text pair weights in the retrieval loss focus learning towards samples from diverse or discrepant neighborhoods: samples where images or text that were close in a semantic space, are distant in the crossmodal space (diversity), or where neighbor relations are asymmetric (discrepancy). Experiments on three challenging datasets exhibiting abstract image-text relations, as well as COCO, demonstrate significant performance gains compared to recent state-of-the-art models and sample weighting approaches.
EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones
Authors: Julien Hauret, Thomas Joubaud, Véronique Zimpfer, Éric Bavu
Date: 2022-10-25 00:00:00
Tasks: Bandwidth Extension, Generative Adversarial Network

In this paper, we present Extreme Bandwidth Extension Network (EBEN), a Generative Adversarial network (GAN) that enhances audio measured with body-conduction microphones. This type of capture equipment suppresses ambient noise at the expense of speech bandwidth, thereby requiring signal enhancement techniques to recover the wideband speech signal. EBEN leverages a multiband decomposition of the raw captured speech to decrease the data time-domain dimensions, and give better control over the full-band signal. This multiband representation is fed to a U-Net-like model, which adopts a combination of feature and adversarial losses to recover an enhanced audio signal. We also benefit from this original representation in the proposed discriminator architecture. Our approach can achieve state-of-the-art results with a lightweight generator and real-time compatible operation.
Semi-Supervised Segmentation of Multi-vendor and Multi-center Cardiac MRI
Authors: Mahyar Bolhassani; Ilkay Oksuz
Date: 2021-05-09 00:00:00
Proceeding: IEEE 2021 5
Tasks: 2D Semantic Segmentation, Segmentation

Automatic segmentation of the heart cavity is an essential task for the diagnosis of cardiac diseases. In this paper, we propose a semi-supervised segmentation setup for leveraging unlabeled data to segment Left-ventricle, Right-ventricle, and Myocardium. We utilize an enhanced version of residual U-Net architecture on a large-scale cardiac MRI dataset. Handling the class imbalanced data issue using dice loss, the enhanced supervised model is able to achieve better dice scores in comparison with a vanilla U-Net model. We applied several augmentation techniques including histogram matching to increase the performance of our model in other domains. Also, we introduce a simple but efficient semi-supervised segmentation method to improve segmentation results without the need for large labeled data. Finally, we applied our method to two benchmark datasets, STACOM2018, and M&Ms 2020 challenges, to show the potency of the proposed model. The effectiveness of our proposed model is demonstrated by the quantitative results. The model achieves average dice scores of 0.921, 0.926, and 0.891 for Left-ventricle, Right-ventricle, and Myocardium respectively.
Comparing Well and Geophysical Data for Temperature Monitoring Within a Bayesian Experimental Design Framework
Authors: Robin Thibaut, Nicolas Compaire, Nolwenn Lesparre, Maximilian Ramgraber, Eric Laloy, Thomas Hermans
Date: 2022-10-19 00:00:00
Proceeding: Water Resources Research 2022 10
Tasks: Experimental Design, Time Series Regression

Temperature logs are an important tool in the geothermal industry. Temperature measurements from boreholes are used for exploration, system design, and monitoring. The number of observations, however, is not always sufficient to fully determine the temperature field or explore the entire parameter space of interest. Drilling in the best locations is still difficult and expensive. It is therefore critical to optimize the number and location of boreholes. Due to its higher spatial resolution and lower cost, four-dimensional (4D) temperature field monitoring via time-lapse Electrical Resistivity Tomography has been investigated as a potential alternative. We use Bayesian Evidential Learning (BEL), a Monte Carlo-based training approach, to optimize the design of a 4D temperature field monitoring experiment. We demonstrate how BEL can take into account various data source combinations (temperature logs combined with geophysical data) in the Bayesian optimal experimental design (BOED). To determine the optimal data source combination, we use the Root Mean Squared Error of the predicted target in the low dimensional latent space where BEL is solving the prediction problem. The parameter estimates are accurate enough to use in BOED. Furthermore, the method is not limited to monitoring temperature fields and can be applied to other similar experimental design problems. The method is computationally efficient and requires little training data. For the considered optimal design problem, a training set of only 200 samples and a test set of 50 samples is sufficient.
A Stacking-Based Model for Non-Invasive Detection of Coronary Heart Disease
Authors: Jikuo Wang; Changchun Liu; Liping Li; Wang Li; Lianke Yao; Han Li; Huan Zhang
Date: 2020-02-20 00:00:00
Proceeding: IEEE Access ( Volume: 8) 2020 2
Tasks: feature selection, Specificity

Coronary arteriongraphy (CAG) is an accurate invasive technique for the diagnosis of coronary heart disease (CHD). However, its invasive procedure is not appropriate for the detection of CHD in the annual physical examination. With the successful application of machine learning (ML) in various fields, our goal is to perform selective integration of multiple ML algorithms and verify the validity of feature selection methods with personal clinical information commonly seen in the annual physical examination. In this study, a two level stacking based model is designed in which level 1 is base-level and level 2 is meta-level. The predictions of base-level classifiers is selected as the input of meta-level. The pearson correlation coefficient and maximum information coefficient are first calculated to find the classifier with the lowest correlation. Then enumeration algorithm is used to find the best combining classifiers which acquire the best result in the end. The Z-Alizadeh Sani CHD dataset which we use consists of 303 cases verified by CAG. Experimental results demonstrate that the proposed model obtains an accuracy, sensitivity and specificity of 95.43%, 95.84%, 94.44%, respectively for the detection of CHD. The proposed method can effectively aid clinicians to detect those with normal coronary arteries from those with CHD.
A Deep Learning Approach for Mobility-Aware and Energy-Efficient Resource Allocation in MEC
Authors: ZAIWAR ALI, SADIA KHAF, ZIAUL HAQ ABBAS, GHULAM ABBAS, FAZAL MUHAMMAD, Sunghwan Kim
Date: 2020-10-01 00:00:00
Proceeding: IEEE Access 2020 10
Tasks: Cloud Computing, Edge-computing

Mobile Edge Computing (MEC) has emerged as an alternative to cloud computing to meet
the latency and Quality-of-Service (QoS) requirements of mobile devices. In this paper, we address the
problem of server resource allocation in MEC. Due to the dynamic load conditions on MEC servers, their
resources need to be used intelligently to meet the QoS requirements of the users and to minimize server
energy consumption. We present a novel resource allocation algorithm, called Power Migration Expand
(PowMigExpand). Our algorithm assigns user requests to the optimal server and allocates optimal amount
of resources to User Equipment (UE) based on our comprehensive utility function. PowMigExpand also
migrates UE requests to new servers, when needed due to the mobility of users. We also present a low cost
Energy Efficient Smart Allocator (EESA) algorithm that uses deep learning for energy efficient allocation
of requests to optimal servers. The proposed algorithms consider varying load of incoming requests and
their heterogeneous nature, energy efficient activation of servers, and Virtual Machine (VM) migration
for smart resource allocation and, thus, is the first comprehensive approach to address the complex and
multidimensional resource allocation problem using deep learning. We compare our proposed algorithms
with other resource allocation approaches and show that our approach can handle the dynamic load conditions
better. The proposed algorithms improve the service rate and the overall utility with minimum energy
consumption. On average, it reduces 26% energy consumption of MESs and improves the service rate by
23%, compared with other algorithms. We also get more than 70% accuracy for EESA in allocating the
resources of multiple servers to multiple users.
Two-part hidden Markov models for semicontinuous longitudinal data with nonignorable missing covariates
Authors: Xiaoxiao Zhou Kai Kang Xinyuan Song
Date: 2020-01-31 00:00:00
Proceeding: Statistics in Medicine. 2020;39:1801–1816. 2020 1
Tasks: Variable Selection

This study develops a two-part hidden Markov model (HMM) for analyzing
semicontinuous longitudinal data in the presence of missing covariates. The pro-
posed model manages a semicontinuous variable by splitting it into two random
variables: a binary indicator for determining the occurrence of excess zeros at
all occasions and a continuous random variable for examining its actual level.
For the continuous longitudinal response, an HMM is proposed to describe the
relationship between the observation and unobservable finite-state transition
processes. The HMM consists of two major components. The first component
is a transition model for investigating how potential covariates influence the
probabilities of transitioning from one hidden state to another. The second
component is a conditional regression model for examining the state-specific
effects of covariates on the response. A shared random effect is introduced to
each part of the model to accommodate possible unobservable heterogeneity
among observation processes and the nonignorability of missing covariates. A
Bayesian adaptive least absolute shrinkage and selection operator (lasso) pro-
cedure is developed to conduct simultaneous variable selection and estimation.
The proposed methodology is applied to a study on the Alzheimer's Disease
Neuroimaging Initiative dataset. New insights into the pathology of Alzheimer's
disease and its potential risk factors are obtained.
A Novel Sparse Bayesian Learning and Its Application to Fault Diagnosis for Multistation Assembly Systems
Authors: Jihoon Chung, Bo Shen, Zhenyu, Kong
Date: 2022-10-28 00:00:00
Tasks: Fault Diagnosis, Time Series Analysis

This paper addresses the problem of fault diagnosis in multistation assembly systems. Fault diagnosis is to identify process faults that cause the excessive dimensional variation of the product using dimensional measurements. For such problems, the challenge is solving an underdetermined system caused by a common phenomenon in practice; namely, the number of measurements is less than that of the process errors. To address this challenge, this paper attempts to solve the following two problems: (1) how to utilize the temporal correlation in the time series data of each process error and (2) how to apply prior knowledge regarding which process errors are more likely to be process faults. A novel sparse Bayesian learning method is proposed to achieve the above objectives. The method consists of three hierarchical layers. The first layer has parameterized prior distribution that exploits the temporal correlation of each process error. Furthermore, the second and third layers achieve the prior distribution representing the prior knowledge of process faults. Then, these prior distributions are updated with the likelihood function of the measurement samples from the process, resulting in the accurate posterior distribution of process faults from an underdetermined system. Since posterior distributions of process faults are intractable, this paper derives approximate posterior distributions via Variational Bayes inference. Numerical and simulation case studies using an actual autobody assembly process are performed to demonstrate the effectiveness of the proposed method.
An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization
Authors: Elvin Lo, Pin-Yu Chen
Date: 2022-10-27 00:00:00

Molecule optimization is an important problem in chemical discovery and has been approached using many techniques, including generative modeling, reinforcement learning, genetic algorithms, and much more. Recent work has also applied zeroth-order (ZO) optimization, a subset of gradient-free optimization that solves problems similarly to gradient-based methods, for optimizing latent vector representations from an autoencoder. In this paper, we study the effectiveness of various ZO optimization methods for optimizing molecular objectives, which are characterized by variable smoothness, infrequent optima, and other challenges. We provide insights on the robustness of various ZO optimizers in this setting, show the advantages of ZO sign-based gradient descent (ZO-signGD), discuss how ZO optimization can be used practically in realistic discovery tasks, and demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite. Code is available at: https://github.com/IBM/QMO-bench.
Multiagent Reinforcement Learning Based on Fusion-Multiactor-Attention-Critic for Multiple-Unmanned-Aerial-Vehicle Navigation Control
Authors: Sangwoo Jeon, Hoeun Lee, Vishnu Kumar Kaliappan, Tuan Anh Nguyen, Hyungeun Jo, Hyeonseo Cho, Dugki Min
Date: 2022-10-10 00:00:00
Proceeding: MDPI Energies 2022 10
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL), Sensor Fusion, Unity

The proliferation of unmanned aerial vehicles (UAVs) has spawned a variety of intelligent services, where efficient coordination plays a significant role in increasing the effectiveness of cooperative execution. However, due to the limited operational time and range of UAVs, achieving highly efficient coordinated actions is difficult, particularly in unknown dynamic environments. This paper proposes a multiagent deep reinforcement learning (MADRL)-based fusion-multiactor-attention-critic (F-MAAC) model for multiple UAVs’ energy-efficient cooperative navigation control. The proposed model is built on the multiactor-attention-critic (MAAC) model, which offers two significant advances. The first is the sensor fusion layer, which enables the actor network to utilize all required sensor information effectively. Next, a layer that computes the dissimilarity weights of different agents is added to compensate for the information lost through the attention layer of the MAAC model. We utilize the UAV LDS (logistic delivery service) environment created by the Unity engine to train the proposed model and verify its energy efficiency. The feature that measures the total distance traveled by the UAVs is incorporated with the UAV LDS environment to validate the energy efficiency. To demonstrate the performance of the proposed model, the F-MAAC model is compared with several conventional reinforcement learning models with two use cases. First, we compare the F-MAAC model to the DDPG, MADDPG, and MAAC models based on the mean episode rewards for 20k episodes of training. The two top-performing models (F-MAAC and MAAC) are then chosen and retrained for 150k episodes. Our study determines the total amount of deliveries done within the same period and the total amount done within the same distance to represent energy efficiency. According to our simulation results, the F-MAAC model outperforms the MAAC model, making 38% more deliveries in 3000 time steps and 30% more deliveries per 1000 m of distance traveled.
Self-supervised clarification question generation for ambiguous multi-turn conversation
Authors: Shao, Taihua;Cai, Fei;Chen, Wanyu;Chen, Honghui
Date: 2021-12-17 00:00:00
Proceeding: journal 2021 12
Tasks: Conversational Recommendation, Question Generation, Question-Generation, Recommendation Systems, Self-Supervised Learning

Clarification Question Generation (CQG) aims to automatically generate clarification questions to avoid misunderstanding. In this paper, we focus on generating clarification questions in the scenario of ambiguous multi-turn conversation, which can be well applied to the interactive systems, e.g., dialogue systems and conversational recommendation systems. As a novel direction, limited manual-annotated samples are available for CQG. Moreover, existing approaches mainly ignore the representation of ambiguous semantics and cannot deal with the Out-of-Vocabulary (OOV) problem in a good manner. To address the above issues, we propose a Self-supervised Hierarchical Pointer-generator model (SHiP) for this task. In detail, similar to the backbone Coarse-to-fine process of CQG, we first formulate two self-supervised learning pretext tasks, i.e., Dialogue History Prediction and Entity Name Prediction. Then, we incorporate a hierarchical Transformer mechanism and a pointer-generator mechanism to understand the ambiguous multi-turn conversations and solve the OOV problem. Finally, we propose an end-to-end co-training paradigm to train the pretext tasks and downstream tasks. We quantify the improvements of SHiP against the competitive baselines on a publicly available dataset CLAQUA, showing a general improvement of 6.75% and 3.91% over state-of-the-art baseline in terms of BLEU and ROUGE-L, respectively.