pwc-archive/papers-with-abstracts huggingface dataset:
Positive Unlabeled Learning Selected Not At Random (PULSNAR): class proportion estimation when the SCAR assumption does not hold
Authors: Praveen Kumar, Christophe G. Lambert
Date: 2023-03-14 00:00:00
Tasks: Binary Classification

Positive and Unlabeled (PU) learning is a type of semi-supervised binary classification where the machine learning algorithm differentiates between a set of positive instances (labeled) and a set of both positive and negative instances (unlabeled). PU learning has broad applications in settings where confirmed negatives are unavailable or difficult to obtain, and there is value in discovering positives among the unlabeled (e.g., viable drugs among untested compounds). Most PU learning algorithms make the \emph{selected completely at random} (SCAR) assumption, namely that positives are selected independently of their features. However, in many real-world applications, such as healthcare, positives are not SCAR (e.g., severe cases are more likely to be diagnosed), leading to a poor estimate of the proportion, $\alpha$, of positives among unlabeled examples and poor model calibration, resulting in an uncertain decision threshold for selecting positives. PU learning algorithms vary; some estimate only the proportion, $\alpha$, of positives in the unlabeled set, while others calculate the probability that each specific unlabeled instance is positive, and some can do both. We propose two PU learning algorithms to estimate $\alpha$, calculate calibrated probabilities for PU instances, and improve classification metrics: i) PULSCAR (positive unlabeled learning selected completely at random), and ii) PULSNAR (positive unlabeled learning selected not at random). PULSNAR employs a divide-and-conquer approach to cluster SNAR positives into subtypes and estimates $\alpha$ for each subtype by applying PULSCAR to positives from each cluster and all unlabeled. In our experiments, PULSNAR outperformed state-of-the-art approaches on both synthetic and real-world benchmark datasets.
Prompting Large Language Models With the Socratic Method
Authors: Edward Y. Chang
Date: 2023-02-17 00:00:00
Tasks: counterfactual, Counterfactual Reasoning, Language Modeling, Language Modelling, Large Language Model

This paper presents a systematic approach to using the Socratic method in developing prompt templates that effectively interact with large language models, including GPT-3. Various methods are examined, and those that yield precise answers and justifications while fostering creativity and imagination to enhance creative writing are identified. Techniques such as {\em definition}, {\em elenchus}, {\em dialectic}, {\em maieutics}, {\em generalization}, and {\em counterfactual reasoning} are discussed for their application in engineering prompt templates and their connections to inductive, deductive, and abductive reasoning. Through examples, the effectiveness of these dialogue and reasoning methods is demonstrated. An interesting observation is made that when the task's goal and user intent are conveyed to GPT-3 via ChatGPT before the start of a dialogue, the large language model seems to connect to the external context expressed in the intent and perform more effectively.
Fashion-model pose recommendation and generation using Machine Learning
Authors: Vijitha Kannumuru, Santhosh Kannan S P, Krithiga Shankar, Joy Larnyoh, Rohith Mahadevan, Raja CSP Raman
Date: 2023-02-19 00:00:00
Tasks: Attribute

Fashion-model pose is an important attribute in the fashion industry. Creative directors, modeling production houses, and top photographers always look for professional models able to pose. without the skill to correctly pose, their chances of landing professional modeling employment are regrettably quite little. There are occasions when models and photographers are unsure of the best pose to strike while taking photographs. This research concentrates on suggesting the fashion personnel a series of similar images based on the input image. The image is segmented into different parts and similar images are suggested for the user. This was achieved by calculating the color histogram of the input image and applying the same for all the images in the dataset and comparing the histograms. Synthetic images have become popular to avoid privacy concerns and to overcome the high cost of photoshoots. Hence, this paper also extends the work of generating synthetic images from the recommendation engine using styleGAN to an extent.
Deep Visual Forced Alignment: Learning to Align Transcription with Talking Face Video
Authors: Minsu Kim, Chae Won Kim, Yong Man Ro
Date: 2023-02-27 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Sentence, speech-recognition, Speech Recognition, Visual Speech Recognition

Forced alignment refers to a technology that time-aligns a given transcription with a corresponding speech. However, as the forced alignment technologies have developed using speech audio, they might fail in alignment when the input speech audio is noise-corrupted or is not accessible. We focus on that there is another component that the speech can be inferred from, the speech video (i.e., talking face video). Since the drawbacks of audio-based forced alignment can be complemented using the visual information when the audio signal is under poor condition, we try to develop a novel video-based forced alignment method. However, different from audio forced alignment, it is challenging to develop a reliable visual forced alignment technology for the following two reasons: 1) Visual Speech Recognition (VSR) has a much lower performance compared to audio-based Automatic Speech Recognition (ASR), and 2) the translation from text to video is not reliable, so the method typically used for building audio forced alignment cannot be utilized in developing visual forced alignment. In order to alleviate these challenges, in this paper, we propose a new method that is appropriate for visual forced alignment, namely Deep Visual Forced Alignment (DVFA). The proposed DVFA can align the input transcription (i.e., sentence) with the talking face video without accessing the speech audio. Moreover, by augmenting the alignment task with anomaly case detection, DVFA can detect mismatches between the input transcription and the input video while performing the alignment. Therefore, we can robustly align the text with the talking face video even if there exist error words in the text. Through extensive experiments, we show the effectiveness of the proposed DVFA not only in the alignment task but also in interpreting the outputs of VSR models.
Active Vision for Deep Visual Learning: A Unified Pooling Framework
Authors: Nan Guo、 Ke Gu、Junfei Qiao、Hantao Liu
Date: 2022-10-10 00:00:00
Proceeding: journal 2022 10

Convolutional neural networks (CNNs) can be generally regarded as learning-based visual systems for computer vision tasks. By imitating the operating mechanism of the human visual system (HVS), CNNs can even achieve better results than human beings in some visual tasks. However, they are primary when compared to the HVS for the reason that the HVS has the ability of active vision to promptly analyze and adapt to specific tasks. In this article, a new unified pooling framework is proposed and a series of pooling methods are designed based on the framework to implement active vision to CNNs. In addition, an active selection pooling (ASP) is put forward to reorganize the existing and newly proposed pooling methods. The CNN models with an ASP tend to have a behavior of focus selection according to tasks during the training process, which acts extremely similar to the HVS.
Self-Organizing Multichannel Deep Learning System for River Turbidity Monitoring
Authors: Ke Gu; Jing Liu; Shuang Shi; Shuangyi Xie; Ting Shi; Junfei Qiao
Date: 2022-12-12 00:00:00
Proceeding: journal 2022 12
Tasks: Deep Learning

This article focuses on proposing a new framework for self-organizing multichannel deep learning system (SMDLS) to solve the problem of river turbidity monitoring, which is one of the most critical indicators of water contamination. First, the basic architecture of our proposed SMDLS is established based on newly designed fractal modules, each of which is composed of one fixed block and one nested block. The nested block in the higher level fractal module is essentially the lower level fractal module. Second, we develop a novel riverway-like generalized regression loss function to improve the robustness of the proposed SMDLS to distorted labels, which is caused by small noise (e.g., the temperature drift noise and wet drift noise of turbidity sensors) in most cases and big noise (e.g., the outlier noise due to solar radiation shielded by mountains) on rare occasions. Finally, we further introduce an ensemble-induced multichannel (EMC) fusion to modify the proposed SMDLS for strengthening its performance and generalization ability. Experiment conducted on large-size river turbidity monitoring database demonstrates that our system considerably outperforms the existing state-of-the-art learning systems.
Air Pollution Prediction in Mass Rallies With a New Temporally-Weighted Sample-Based Multitask Learner
Authors: Ke Gu; Jing Liu; Shuang Shi; Shuangyi Xie; Ting Shi; Junfei Qiao
Date: 2022-03-09 00:00:00
Proceeding: journal 2022 3
Tasks: Air Pollution Prediction, Prediction

With the rapid development of industrialization, the environmental pollution issue is becoming increasingly serious, especially the air pollution problem. As the core of the prevention and control of air pollution, air pollution prediction plays a very significant role in human survival and development. Therefore, it is highly essential to develop an accurate air pollution prediction model for mass rallies (e.g., playground and bazaar). Recent studies have suggested that multiple air contaminants, e.g., PM 2.5 and PM 10 , which belong to a kind of aerosol, can carry the Covid-19 virus and spread it rapidly through the atmosphere, and this dramatically increases the risk of Covid-19 infection, particularly in the crowded and enclosed environment. Nevertheless, most existing air pollution prediction methods, which rely on large amounts of historical data for modeling and assume that the crowd flows relatively slow, are difficult to apply well to predict air pollution in mass rallies. To solve the aforementioned problem and better assist the decision-makers in managing environmental risk to human beings, in this article, we come up with a novel air pollution prediction model for mass rallies. More specifically, we first propose a temporally weighting matrix to differentiate the significance of training samples in the time domain. Then, we construct a temporal support vector regressor (TSVR), which puts more emphasis on the adjacent samples by considering the fact that the crowd usually flows promptly and disorderly in mass rallies. Finally, based on the extended TSVR, we develop a multitask TSVR (MTSVR) that simultaneously considers the related tasks. Since different air contaminants are correlated with each other, all the tasks can benefit by sharing information. The results of comparison experiments demonstrate that our presented MTSVR outperforms state-of-the-art single-task learners, multitask learners, and air pollution predictors when applied for air pollution prediction in mass rallies. Particularly, when under the six-task condition, the error values of the prediction of PM 2.5 , PM 10 , and O 3 obtained by our proposed method are relatively lower, outperforming the most advanced method tested by 15.2%, 6.1%, and 4.3%, and the precision values of the predicted values outperform the advanced method tested by 28.3%, 25.1%, and 24.8%.
APRIL: Active Preference-learning based Reinforcement Learning
Authors: Riad Akrour, Marc Schoenauer, Michèle Sebag
Date: 2012-08-05 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper focuses on reinforcement learning (RL) with limited prior knowledge. In the domain of swarm robotics for instance, the expert can hardly design a reward function or demonstrate the target behavior, forbidding the use of both standard RL and inverse reinforcement learning. Although with a limited expertise, the human expert is still often able to emit preferences and rank the agent demonstrations. Earlier work has presented an iterative preference-based RL framework: expert preferences are exploited to learn an approximate policy return, thus enabling the agent to achieve direct policy search. Iteratively, the agent selects a new candidate policy and demonstrates it; the expert ranks the new demonstration comparatively to the previous best one; the expert's ranking feedback enables the agent to refine the approximate policy return, and the process is iterated. In this paper, preference-based reinforcement learning is combined with active ranking in order to decrease the number of ranking queries to the expert needed to yield a satisfactory policy. Experiments on the mountain car and the cancer treatment testbeds witness that a couple of dozen rankings enable to learn a competent policy.
CUR Transformer: A Convolutional Unbiased Regional Transformer for Image Denoising
Authors: Kang Xu, Weixin Li, Xia Wang, Xiaoyan Hu, Ke Yan, Xiaojie Wang, Xuan Dong
Date: 2023-02-25 00:00:00
Proceeding: journal 2023 2
Tasks: Denoising, Image Denoising, Image Enhancement, Jpeg Compression Artifact Reduction, Low-Light Image Enhancement

Image denoising is a fundamental problem in computer vision and multimedia computation. Non-local filters are effective for image denoising. But existing deep learning methods that use non-local computation structures are mostly designed for high-level tasks, and global self-attention is usually adopted. For the task of image denoising, they have high computational complexity and have a lot of redundant computation of uncorrelated pixels. To solve this problem and combine the marvelous advantages of non-local filter and deep learning, we propose a Convolutional Unbiased Regional (CUR) transformer. Based on the prior that, for each pixel, its similar pixels are usually spatially close, our insights are that (1) we partition the image into non-overlapped windows and perform regional self-attention to reduce the search range of each pixel, and (2) we encourage pixels across different windows to communicate with each other. Based on our insights, the CUR transformer is cascaded by a series of convolutional regional self-attention (CRSA) blocks with U-style short connections. In each CRSA block, we use convolutional layers to extract the query, key, and value features, namely Q, K, and V, of the input feature. Then, we partition the Q, K, and V features into local non-overlapped windows and perform regional self-attention within each window to obtain the output feature of this CRSA block. Among different CRSA blocks, we perform the unbiased window partition by changing the partition positions of the windows. Experimental results show that the CUR transformer outperforms the state-of-the-art methods significantly on four low-level vision tasks, including real and synthetic image denoising, JPEG compression artifact reduction, and low-light image enhancement.
A Low Rank Weighted Graph Convolutional Approach to Weather Prediction
Authors: Tyler Wilson, Pang-Ning Tan, Lifeng Luo
Date: 2018-11-17 00:00:00
Proceeding: IEEE 2018 11
Tasks: Weather Forecasting

Weather forecasting is an important but challenging problem as one must contend with the inherent non-linearities and spatiotemporal autocorrelation present in the data. This paper presents a novel deep learning approach based on a coupled weighted graph convolutional LSTM (WGC-LSTM) to address these challenges. Specifically, our proposed approach uses an LSTM to capture the inherent temporal autocorrelation of the data and a graph convolution to model its spatial relationships. As the weather condition can be influenced by various spatial factors besides the distance between locations, e.g., topography, prevailing winds and jet streams, imposing a fixed graph structure based on the proximity between locations is insufficient to train a robust deep learning model. Instead, our proposed approach treats the adjacency matrix of the graph as a model parameter that can be learned from the training data. However, this introduces an additional O(|V |2) parameters to be estimated, where V is the number of locations. With large graphs this may also lead to slower performance as well as susceptibility to overfitting. We propose a modified version of our approach that can address this difficulty by assuming that the adjacency matrix is either sparse or low rank. Experimental results using two real-world weather datasets show that WGCLSTM outperforms all other baseline methods for the majority of the evaluated locations.
Semantic Structure Enhanced Contrastive Adversarial Hash Network for Cross-media Representation Learning
Authors: Meiyu Liang, Junping Du, Xiaowen Cao, Yang Yu, Kangkang Lu, Zhe Xue, Min Zhang
Date: 2022-10-22 00:00:00
Proceeding: ACM Multimedia 2022 10
Tasks: Representation Learning

Deep cross-media hashing technology provides an efficient cross-media representation learning solution for cross-media search. However, the existing methods do not consider both fine-grained semantic features and semantic structures to mine implicit cross-media semantic associations, which leads to weaker semantic discrimination and consistency for cross-media representation. To tackle this problem, we propose a novel semantic structure enhanced contrastive adversarial hash network for cross-media representation learning (SCAHN). Firstly, in order to capture more fine-grained cross-media semantic associations, a fine-grained cross-media attention feature learning network is constructed, thus the learned saliency features of different modalities are more conducive to cross-media semantic alignment and fusion. Secondly, for further improving learning ability of implicit cross-media semantic associations, a semantic label association graph is constructed, and the graph convolutional network is utilized to mine the implicit semantic structures, thus guiding learning of discriminative features of different modalities. Thirdly, a cross-media and intra-media contrastive adversarial representation learning mechanism is proposed to further enhance the semantic discriminativeness of different modal representations, and a dual-way adversarial learning strategy is developed to maximize cross-media semantic associations, so as to obtain cross-media unified representations with stronger discriminativeness and semantic consistency preserving power. Extensive experiments on several cross-media benchmark datasets demonstrate that the proposed SCAHN outperforms the state-of-the-art methods.
CoLo-CAM: Class Activation Mapping for Object Co-Localization in Weakly-Labeled Unconstrained Videos
Authors: Soufiane Belharbi, Shakeeb Murtaza, Marco Pedersoli, Ismail Ben Ayed, Luke McCaffrey, Eric Granger
Date: 2023-03-16 00:00:00
Tasks: Object, Object Localization

Leveraging spatiotemporal information in videos is critical for weakly supervised video object localization (WSVOL) tasks. However, state-of-the-art methods only rely on visual and motion cues, while discarding discriminative information, making them susceptible to inaccurate localizations. Recently, discriminative models have been explored for WSVOL tasks using a temporal class activation mapping (CAM) method. Although their results are promising, objects are assumed to have limited movement from frame to frame, leading to degradation in performance for relatively long-term dependencies. This paper proposes a novel CAM method for WSVOL that exploits spatiotemporal information in activation maps during training without constraining an object's position. Its training relies on Co-Localization, hence, the name CoLo-CAM. Given a sequence of frames, localization is jointly learned based on color cues extracted across the corresponding maps, by assuming that an object has similar color in consecutive frames. CAM activations are constrained to respond similarly over pixels with similar colors, achieving co-localization. This improves localization performance because the joint learning creates direct communication among pixels across all image locations and over all frames, allowing for transfer, aggregation, and correction of localizations. Co-localization is integrated into training by minimizing the color term of a conditional random field (CRF) loss over a sequence of frames/CAMs. Extensive experiments on two challenging YouTube-Objects datasets of unconstrained videos show the merits of our method, and its robustness to long-term dependencies, leading to new state-of-the-art performance for WSVOL task.
NAISR: A 3D Neural Additive Model for Interpretable Shape Representation
Authors: Yining Jiao, Carlton Zdanski, Julia Kimbell, Andrew Prince, Cameron Worden, Samuel Kirse, Christopher Rutter, Benjamin Shields, William Dunn, Jisan Mahmud, Marc Niethammer
Date: 2023-03-16 00:00:00
Tasks: 3D Shape Reconstruction, Disentanglement, Hippocampus

Deep implicit functions (DIFs) have emerged as a powerful paradigm for many computer vision tasks such as 3D shape reconstruction, generation, registration, completion, editing, and understanding. However, given a set of 3D shapes with associated covariates there is at present no shape representation method which allows to precisely represent the shapes while capturing the individual dependencies on each covariate. Such a method would be of high utility to researchers to discover knowledge hidden in a population of shapes. For scientific shape discovery, we propose a 3D Neural Additive Model for Interpretable Shape Representation ($\texttt{NAISR}$) which describes individual shapes by deforming a shape atlas in accordance to the effect of disentangled covariates. Our approach captures shape population trends and allows for patient-specific predictions through shape transfer. $\texttt{NAISR}$ is the first approach to combine the benefits of deep implicit shape representations with an atlas deforming according to specified covariates. We evaluate $\texttt{NAISR}$ with respect to shape reconstruction, shape disentanglement, shape evolution, and shape transfer on three datasets: 1) $\textit{Starman}$, a simulated 2D shape dataset; 2) the ADNI hippocampus 3D shape dataset; and 3) a pediatric airway 3D shape dataset. Our experiments demonstrate that $\textit{Starman}$ achieves excellent shape reconstruction performance while retaining interpretability. Our code is available at $\href{https://github.com/uncbiag/NAISR}{https://github.com/uncbiag/NAISR}$.
A Dual Branch Network for Emotional Reaction Intensity Estimation
Authors: Jun Yu, Jichao Zhu, Wangyuan Zhu, Zhongpeng Cai, Guochen Xie, Renda Li, Gongpeng Zhao
Date: 2023-03-16 00:00:00
Tasks: regression

Emotional Reaction Intensity(ERI) estimation is an important task in multimodal scenarios, and has fundamental applications in medicine, safe driving and other fields. In this paper, we propose a solution to the ERI challenge of the fifth Affective Behavior Analysis in-the-wild(ABAW), a dual-branch based multi-output regression model. The spatial attention is used to better extract visual features, and the Mel-Frequency Cepstral Coefficients technology extracts acoustic features, and a method named modality dropout is added to fusion multimodal features. Our method achieves excellent results on the official validation set.