pwc-archive/papers-with-abstracts huggingface dataset:
Multi-objective Learning to Rank by Model Distillation
Authors: Jie Tang, Huiji Gao, Liwei He, Sanjeev Katariya
Date: 2024-07-09 00:00:00
Tasks: Learning-To-Rank, model

In online marketplaces, search ranking's objective is not only to purchase or conversion (primary objective), but to also the purchase outcomes(secondary objectives), e.g. order cancellation(or return), review rating, customer service inquiries, platform long term growth. Multi-objective learning to rank has been widely studied to balance primary and secondary objectives. But traditional approaches in industry face some challenges including expensive parameter tuning leads to sub-optimal solution, suffering from imbalanced data sparsity issue, and being not compatible with ad-hoc objective. In this paper, we propose a distillation-based ranking solution for multi-objective ranking, which optimizes the end-to-end ranking system at Airbnb across multiple ranking models on different objectives along with various considerations to optimize training and serving efficiency to meet industry standards. We found it performs much better than traditional approaches, it doesn't only significantly increases primary objective by a large margin but also meet secondary objectives constraints and improve model stability. We also demonstrated the proposed system could be further simplified by model self-distillation. Besides this, we did additional simulations to show that this approach could also help us efficiently inject ad-hoc non-differentiable business objective into the ranking system while enabling us to balance our optimization objectives.
Knowledge graph-enhanced molecular contrastive learning with functional prompt
Authors: Yin Fang, Qiang Zhang, Ningyu Zhang, Zhuo Chen, Xiang Zhuang, Xin Shao, Xiaohui Fan & Huajun Chen
Date: 2023-05-04 00:00:00
Proceeding: journal 2023 5
Tasks: Contrastive Learning, Drug Design, Molecular Property Prediction, molecular representation, Property Prediction

Deep learning models can accurately predict molecular properties and help making the search for potential drug candidates faster and more efficient. Many existing methods are purely data driven, focusing on exploiting the intrinsic topology and construction rules of molecules without any chemical prior information. The high data dependency makes them difficult to generalize to a wider chemical space and leads to a lack of interpretability of predictions. Here, to address this issue, we introduce a chemical element-oriented knowledge graph to summarize the basic knowledge of elements and their closely related functional groups. We further propose a method for knowledge graph-enhanced molecular contrastive learning with functional prompt (KANO), exploiting external fundamental domain knowledge in both pre-training and fine-tuning. Specifically, with element-oriented knowledge graph as a prior, we first design an element-guided graph augmentation in contrastive-based pre-training to explore microscopic atomic associations without violating molecular semantics. Then, we learn functional prompts in fine-tuning to evoke the downstream task-related knowledge acquired by the pre-trained model. Extensive experiments show that KANO outperforms state-of-the-art baselines on 14 molecular property prediction datasets and provides chemically sound explanations for its predictions. This work contributes to more efficient drug design by offering a high-quality knowledge prior, interpretable molecular representation and superior prediction performance.
Multi-level Protein Structure Pre-training via Prompt Learning
Authors: Zeyuan Wang, Qiang Zhang, Shuang-Wei HU, Haoran Yu, Xurui Jin, Zhichen Gong, Huajun Chen
Date: 2023-02-17 00:00:00
Proceeding: journal 2023 2
Tasks: Prediction, Prompt Learning

A protein can focus on different structure levels to implement its functions. Each structure has its own merit and driving forces in describing some specific characteristics, and they cannot replace each other. Most existing function prediction methods take the tertiary structure as input, unintentionally ignoring the other levels of protein structures. Considering protein sequences can determine multi-level structures, in this paper, we aim to realize the comprehensive potential of protein sequences for function prediction. Specifically, we propose a new prompt-guided multi-task pre-training and fine-tuning framework, and the resulting protein model is called PromptProtein. Through the prompt-guided multi-task pre-training, we learn multiple prompt signals to steer the model to focus on different structure levels. We also design a prompt fine-tuning module to provide downstream tasks the on-demand flexibility of utilizing respective levels of structure information. Extensive experiments on function prediction and protein engineering show that PromptProtein outperforms state-of-the-art methods by large margins.
Adaptive Decoupled Pose Knowledge Distillation
Authors: Jie Xu, Shanshan Zhang, and Jian Yang
Date: 2023-10-01 00:00:00
Proceeding: journal 2023 10
Tasks: Knowledge Distillation, Pose Estimation

Existing state-of-the-art human pose estimation approaches require heavy computational resources for accurate prediction. One promising technique to obtain an accurate yet lightweight pose estimator is Knowledge Distillation (KD), which distills the pose knowledge from a powerful teacher model to a lightweight student model. However, existing human pose KD methods focus more on designing paired student and teacher network architectures, yet ignore the mechanism of pose knowledge distillation. In this work, we reformulate the human pose KD to a coarse to fine process and decouple the classical KD loss into three terms: Binary Keypoint vs. Non-Keypoint Distillation (BiKD), Keypoint Area Distillation (KAD) and Non-keypoint Area Distillation (NAD). Observing the decoupled formulation, we point out an important limitation of the classical pose KD, i.e. the bias between different loss terms limits the performance gain of the student network. To address the biased knowledge distillation problem, we present a novel KD method named Adaptive Decoupled Pose knowledge Distillation (ADPD), enabling BiKD, KAD and NAD to play their roles more effectively and flexibly. Extensive experiments on two standard human pose datasets, MPII and MS COCO, demonstrate that our proposed method outperforms previous KD methods and is generalizable to different teacher-student pairs. The code will be available at https://github.com/SuperJay1996/ADPD.
OKGR: Occluded Keypoint Generation and Refinement for 3D Object Detection
Authors: Mingqian Ji, Jian Yang & Shanshan Zhang
Date: 2023-12-23 00:00:00
Proceeding: journal 2023 12
Tasks: 3D Object Detection, Autonomous Driving, object-detection, Object Detection

Lidar-based 3D object detectors utilize point clouds to detect objects in autonomous driving. However, the point clouds are sparse and incomplete, which affects the detectors’ learning of shape knowledge and limits the 3D detection performance. Previous works improve performance through completing object shape at the point level or representation level, such as voxel. The former increases computational burden, while the latter has poor generalization ability to point-based detectors. In this paper, we present an approach, namely Occluded Keypoint Generation and Refinement (OKGR), which is effective to improve 3D detection performance by completing object features at the keypoint level. Specifically, Occluded Keypoint Generation (OKG) generates occluded keypoints to densify raw keypoints and learns the offsets between the generated keypoints and prototypes, while retaining the raw keypoints unchanged. Occluded Keypoint Refinement (OKR) assigns weights to the generated keypoints and conducts these weights to features to obtain high-quality complete features for detection. We apply our approach to two representative detectors, PV-RCNN++ and PDV, and evaluate the detectors on KITTI and Waymo Open Dataset. The experiments show significant performance improvement. Particularly, our OKGR applied on PV-RCNN++ achieves improvements of Pedestrian and Cyclist of +3.19%, +2.53% AP on average difficulty levels on KITTI, and +2.18%, +2.29% mAPH on Waymo Open Dataset. For more information, the supplementary material and code are available at https://github.com/Mingqj/OKGR.
Multimodality Invariant Learning for Multimedia-Based New Item Recommendation
Authors: Haoyue Bai, Le Wu, Min Hou, Miaomiao Cai, Zhuangzhuang He, Yuyang Zhou, Richang Hong, Meng Wang
Date: 2024-04-28 00:00:00

Multimedia-based recommendation provides personalized item suggestions by learning the content preferences of users. With the proliferation of digital devices and APPs, a huge number of new items are created rapidly over time. How to quickly provide recommendations for new items at the inference time is challenging. What's worse, real-world items exhibit varying degrees of modality missing(e.g., many short videos are uploaded without text descriptions). Though many efforts have been devoted to multimedia-based recommendations, they either could not deal with new multimedia items or assumed the modality completeness in the modeling process. In this paper, we highlight the necessity of tackling the modality missing issue for new item recommendation. We argue that users' inherent content preference is stable and better kept invariant to arbitrary modality missing environments. Therefore, we approach this problem from a novel perspective of invariant learning. However, how to construct environments from finite user behavior training data to generalize any modality missing is challenging. To tackle this issue, we propose a novel Multimodality Invariant Learning reCommendation(a.k.a. MILK) framework. Specifically, MILK first designs a cross-modality alignment module to keep semantic consistency from pretrained multimedia item features. After that, MILK designs multi-modal heterogeneous environments with cyclic mixup to augment training data, in order to mimic any modality missing for invariant user preference learning. Extensive experiments on three real datasets verify the superiority of our proposed framework. The code is available at https://github.com/HaoyueBai98/MILK.
Explainable Survival Analysis with Uncertainty using Convolution-Involved Vision Transformer
Authors: Zhihao Tang, Li Liu, Yifan Shen, Zongyi Chen, Guixiang Ma, Jiyan Dong, Xujie Sun, Xi Zhang, Chaozhuo Li, Qingfeng Zheng, Lin Yang
Date: 2024-04-18 00:00:00
Proceeding: journal 2024 4
Tasks: Survival Analysis, Survival Prediction

Highlights•Without patching WSIs, a novel ViT-based model is proposed for survival predictions.•We first introduce aleatoric uncertainty into the survival loss function.•We explain survival prediction using a post-hoc explainable method.•Our method outperforms baselines in accuracy, explainability, and reliability.
SPEFL: Efficient Security and Privacy Enhanced Federated Learning Against Poisoning Attacks
Authors: Zhenhan Ke、Liyan Shen、Jinqiao Shi、Xi Zhang
Date: 2023-01-01 00:00:00
Proceeding: journal 2023 1
Tasks: Federated Learning, Privacy Preserving

Federated learning (FL) is a distributed machine learning paradigm in the Internet of Things (IoT), which allows multiple devices to collaboratively train models without leaking local data. In the open scenario of IoT, malicious devices can launch poisoning attacks to compromise the final model by submitting crafted gradients. Some previous studies defend against poisoning attacks by analyzing the statistical characteristics of plaintext gradients. However, plaintext gradients would expose private information to malicious FL devices or servers. To simultaneously resist poisoning attacks and preserve privacy, cryptography technology can be utilized to obfuscate the gradients in defense methods, but the private calculation of resisting poisoning attack methods will cause efficiency problems, especially imposing unaffordable overhead on resource-limited IoT devices. Therefore, resisting poisoning attacks efficiently while protecting privacy remains a challenge. This paper proposes a secure and privacy-enhanced FL (SPEFL) framework for efficient privacy-preserving and poisoning-resistant federated learning in IoT. We design an efficient secure computation protocol based on a three-server architecture to facilitate the cryptographic computation of large linear and complex nonlinear operators in the method against poisoning attacks. In SPEFL, most of the calculations are efficiently performed on the servers, which will not impose too much burden on resource-limited IoT devices. In addition, we design a security-enhanced verifiable protocol to detect the malicious behavior of the server and guarantee the correctness of FL aggregation results. Experimental and theoretical results demonstrate that SPEFL can efficiently complete FL training meanwhile guaranteeing the accuracy of the model.
SPCTNet: A Series-Parallel CNN and Transformer Network for 3D Medical Image Segmentation
Authors: Bin Yu, Quan Zhou & Xuming Zhang
Date: 2024-02-04 00:00:00
Proceeding: journal 2024 2
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Medical image segmentation is crucial for lesion localization and surgical navigation. Recent advancements in medical image segmentation have been driven by Convolutional Neural Networks (CNNs) and Transformers. However, CNNs have limitations in capturing long-range dependencies due to their weight sharing and localized receptive fields, posing challenges in handling varying organ shapes. While Transformers offer an alternative with global receptive fields, their spatial and computational complexity is particularly high, especially for 3D medical images. To address this issue, we propose a novel series-parallel network that combines convolution and self-attention for 3D medical image segmentation. We utilize a serial 3D CNN as the encoder to extract multi-level feature maps, which are fused via a feature pyramid network. Subsequently, we adopt four parallel Transformer branches to capture global features. To efficiently model long-range information, we introduce patch self-attention, which divides the input into non-overlapping patches and computes attention between corresponding pixels across patches. Experimental evaluations on 3D MRI prostate and left atrial segmentation tasks confirm the superior performance of our network compared to other CNN and Transformer-based networks. Notably, our method achieves higher segmentation accuracy and faster inference speed.
The Balanced-Pairwise-Affinities Feature Transform
Authors: Daniel Shalam, Simon Korman
Date: 2024-06-25 00:00:00
Tasks: Few-Shot Image Classification, Image Clustering, Person Re-Identification

The Balanced-Pairwise-Affinities (BPA) feature transform is designed to upgrade the features of a set of input items to facilitate downstream matching or grouping related tasks. The transformed set encodes a rich representation of high order relations between the input features. A particular min-cost-max-flow fractional matching problem, whose entropy regularized version can be approximated by an optimal transport (OT) optimization, leads to a transform which is efficient, differentiable, equivariant, parameterless and probabilistically interpretable. While the Sinkhorn OT solver has been adapted extensively in many contexts, we use it differently by minimizing the cost between a set of features to $itself$ and using the transport plan's $rows$ as the new representation. Empirically, the transform is highly effective and flexible in its use and consistently improves networks it is inserted into, in a variety of tasks and training schemes. We demonstrate state-of-the-art results in few-shot classification, unsupervised image clustering and person re-identification. Code is available at \url{github.com/DanielShalam/BPA}.
Trainable Fractional Fourier Transform
Authors: Emirhan Koç, Tuna Alikaşifoğlu, Arda Can Aras, Aykut Koç
Date: 2024-03-04 00:00:00
Proceeding: IEEE Signal Processing Letters 2024 3
Tasks: image-classification, Image Classification, Time Series, Time Series Prediction

Recently, the fractional Fourier transform (FrFT) has been integrated into distinct deep neural network (DNN) models such as transformers, sequence models, and convolutional neural networks (CNNs). In these studies, the fraction order is considered a hyperparameter and tuned manually to find the suitable values. By taking these one step further, we extend the scope of FrFT and introduce it as a trainable layer in various neural network architectures, where the fraction order is learned in the training stage along with the network weights. First, we mathematically show that fraction order can be updated through backpropagation in the network training phase. To support this formulation, we conduct extensive experiments encompassing image classification and time series prediction tasks on benchmark datasets. Our results show that the trainable FrFT layers alleviate the need to search for suitable fraction orders and improve performance over time and Fourier domain approaches.
Dual Circle Contrastive Learning-Based Blind Image Super-Resolution
Authors: Y Qiu，Q Zhu，S Zhu，B Zeng
Date: 2023-03-29 00:00:00
Proceeding: journal 2023 3
Tasks: Contrastive Learning, Image Super-Resolution, Super-Resolution

Blind image super-resolution (BISR) aims to construct high-resolution image from low-resolution (LR) image that contains unknown degradation. Although the previous methods demonstrated impressive performance by introducing the degradation representation in BISR task, there still exist two problems in most of them. First, they ignore the degradation characteristics of different image regions when generating degradation representation. Second, they lack effective supervision on the generation of both degradation representation and super-resolution result. To solve these problems, we propose the dual circle contrastive learning (DCCL) with the high-efficiency modules to implement BISR. In our proposed method, we design the degradation extraction network to obtain the degradation representations from different texture regions of LR image. Meanwhile, we propose DCCL coupled with the degrading network to guarantee the obtained degradation representation to contain the degradation of LR image as much as possible. The application of DCCL also makes the SR results contain
Flexible Fusion Network for Multi-modal Brain Tumor Segmentation
Authors: Hengyi Yang; Tao Zhou; Yi Zhou; Yizhe Zhang; Huazhu Fu
Date: 2023-05-01 00:00:00
Proceeding: journal 2023 5
Tasks: Brain Tumor Segmentation, Decoder, Segmentation, Tumor Segmentation

Automated brain tumor segmentation is crucial for aiding brain disease diagnosis and evaluating disease progress. Currently, magnetic resonance imaging (MRI) is a routinely adopted approach in the field of brain tumor segmentation that can provide different modality images. It is critical to leverage multi-modal images to boost brain tumor segmentation performance. Existing works commonly concentrate on generating a shared representation by fusing multi-modal data, while few methods take into account modality-specific characteristics. Besides, how to efficiently fuse arbitrary numbers of modalities is still a difficult task. In this study, we present a flexible fusion network (termed F 2 Net) for multi-modal brain tumor segmentation, which can flexibly fuse arbitrary numbers of multi-modal information to explore complementary information while maintaining the specific characteristics of each modality. Our F 2 Net is based on the encoder-decoder structure, which utilizes two Transformer-based feature learning streams and a cross-modal shared learning network to extract individual and shared feature representations. To effectively integrate the knowledge from the multi-modality data, we propose a cross-modal feature-enhanced module (CFM) and a multi-modal collaboration module (MCM), which aims at fusing the multi-modal features into the shared learning network and incorporating the features from encoders into the shared decoder, respectively. Extensive experimental results on multiple benchmark datasets demonstrate the effectiveness of our F 2 Net over other state-of-the-art segmentation methods.
Catch: Collaborative Feature Set Search for Automated Feature Engineering
Authors: Guoshan Lu, Haobo Wang, Saisai Yang, Jing Yuan, Guozheng Yang, Cheng Zang, Gang Chen, and Junbo ZhaoAuthors Info & Claims
Date: 2023-04-30 00:00:00
Proceeding: journal 2023 4
Tasks: Automated Feature Engineering, Decision Making, Feature Engineering, reinforcement-learning, Reinforcement Learning

Feature engineering often plays a crucial role in building mining systems for tabular data, which traditionally requires experienced human experts to perform. Thanks to the rapid advances in reinforcement learning, it has offered an automated alternative, i.e. automated feature engineering (AutoFE). In this work, through scrutiny of the prior AutoFE methods, we characterize several research challenges that remained in this regime, concerning system-wide efficiency, efficacy, and practicality toward production. We then propose Catch, a full-fledged new AutoFE framework that comprehensively addresses the aforementioned challenges. The core to Catch composes a hierarchical-policy reinforcement learning scheme that manifests a collaborative feature engineering exploration and exploitation grounded on the granularity of the whole feature set. At a higher level of the hierarchy, a decision-making module controls the post-processing of the attained feature engineering transformation. We extensively experiment with Catch on 26 academic standardized tabular datasets and 9 industrialized real-world datasets. Measured by numerous metrics and analyses, Catch establishes a new state-of-the-art, from perspectives performance, latency as well as its practicality towards production. Source code1 can be found at https://github.com/1171000709/Catch.
Learning a Data-Driven Policy Network for Pre-Training Automated Feature Engineering
Authors: Liyao Li, Haobo Wang, Liangyu Zha, Qingyi Huang, Sai Wu, Gang Chen, Junbo Zhao
Date: 2023-02-02 00:00:00
Proceeding: Conference 2023 2
Tasks: Automated Feature Engineering, Feature Engineering, Neural Architecture Search

Feature engineering is widely acknowledged to be pivotal in tabular data analysis and prediction. Automated feature engineering (AutoFE) emerged to automate this process managed by experienced data scientists and engineers conventionally. In this area, most — if not all — prior work adopted an identical framework from the neural architecture search (NAS) method. While feasible, we posit that the NAS framework very much contradicts the way how human experts cope with the data since the inherent Markov decision process (MDP) setup differs. We point out that its data-unobserved setup consequentially results in an incapability to generalize across different datasets as well as also high computational cost. This paper proposes a novel AutoFE framework Feature Set Data-Driven Search (FETCH), a pipeline mainly for feature generation and selection. Notably, FETCH is built on a brand-new data-driven MDP setup using the tabular dataset as the state fed into the policy network. Further, we posit that the crucial merit of FETCH is its transferability where the yielded policy network trained on a variety of datasets is indeed capable to enact feature engineering on unseen data, without requiring additional exploration. To the best of our knowledge, this is a pioneer attempt to build a tabular data pre-training paradigm via AutoFE. Extensive experiments show that FETCH systematically surpasses the current state-of-the-art AutoFE methods and validates the transferability of AutoFE pre-training.