pwc-archive/papers-with-abstracts huggingface dataset:
IBoxCLA: Towards Robust Box-supervised Segmentation of Polyp via Improved Box-dice and Contrastive Latent-anchors
Authors: Zhiwei Wang, Qiang Hu, Hongkuan Shi, Li He, Man He, Wenxuan Dai, Yinjiao Tian, Xin Yang, Mei Liu, Qiang Li
Date: 2023-10-11 00:00:00
Tasks: Segmentation

Box-supervised polyp segmentation attracts increasing attention for its cost-effective potential. Existing solutions often rely on learning-free methods or pretrained models to laboriously generate pseudo masks, triggering Dice constraint subsequently. In this paper, we found that a model guided by the simplest box-filled masks can accurately predict polyp locations/sizes, but suffers from shape collapsing. In response, we propose two innovative learning fashions, Improved Box-dice (IBox) and Contrastive Latent-Anchors (CLA), and combine them to train a robust box-supervised model IBoxCLA. The core idea behind IBoxCLA is to decouple the learning of location/size and shape, allowing for focused constraints on each of them. Specifically, IBox transforms the segmentation map into a proxy map using shape decoupling and confusion-region swapping sequentially. Within the proxy map, shapes are disentangled, while locations/sizes are encoded as box-like responses. By constraining the proxy map instead of the raw prediction, the box-filled mask can well supervise IBoxCLA without misleading its shape learning. Furthermore, CLA contributes to shape learning by generating two types of latent anchors, which are learned and updated using momentum and segmented polyps to steadily represent polyp and background features. The latent anchors facilitate IBoxCLA to capture discriminative features within and outside boxes in a contrastive manner, yielding clearer boundaries. We benchmark IBoxCLA on five public polyp datasets. The experimental results demonstrate the competitive performance of IBoxCLA compared to recent fully-supervised polyp segmentation methods, and its superiority over other box-supervised state-of-the-arts with a relative increase of overall mDice and mIoU by at least 6.5% and 7.5%, respectively.
Optimizing the Placement of Roadside LiDARs for Autonomous Driving
Authors: Wentao Jiang, Hao Xiang, Xinyu Cai, Runsheng Xu, Jiaqi Ma, Yikang Li, Gim Hee Lee, Si Liu
Date: 2023-10-11 00:00:00
Proceeding: ICCV 2023 1
Tasks: Autonomous Driving

Multi-agent cooperative perception is an increasingly popular topic in the field of autonomous driving, where roadside LiDARs play an essential role. However, how to optimize the placement of roadside LiDARs is a crucial but often overlooked problem. This paper proposes an approach to optimize the placement of roadside LiDARs by selecting optimized positions within the scene for better perception performance. To efficiently obtain the best combination of locations, a greedy algorithm based on perceptual gain is proposed, which selects the location that can maximize the perceptual gain sequentially. We define perceptual gain as the increased perceptual capability when a new LiDAR is placed. To obtain the perception capability, we propose a perception predictor that learns to evaluate LiDAR placement using only a single point cloud frame. A dataset named Roadside-Opt is created using the CARLA simulator to facilitate research on the roadside LiDAR placement problem.
SAGE-ICP: Semantic Information-Assisted ICP
Authors: Jiaming Cui, Jiming Chen, Liang Li
Date: 2023-10-11 00:00:00
Tasks: Pose Estimation

Robust and accurate pose estimation in unknown environments is an essential part of robotic applications. We focus on LiDAR-based point-to-point ICP combined with effective semantic information. This paper proposes a novel semantic information-assisted ICP method named SAGE-ICP, which leverages semantics in odometry. The semantic information for the whole scan is timely and efficiently extracted by a 3D convolution network, and these point-wise labels are deeply involved in every part of the registration, including semantic voxel downsampling, data association, adaptive local map, and dynamic vehicle removal. Unlike previous semantic-aided approaches, the proposed method can improve localization accuracy in large-scale scenes even if the semantic information has certain errors. Experimental evaluations on KITTI and KITTI-360 show that our method outperforms the baseline methods, and improves accuracy while maintaining real-time performance, i.e., runs faster than the sensor frame rate.
AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation
Authors: Liyang Chen, Weihong Bao, Shun Lei, Boshi Tang, Zhiyong Wu, Shiyin Kang, HaoZhi Huang, Helen Meng
Date: 2023-10-11 00:00:00

Speech-driven 3D facial animation aims at generating facial movements that are synchronized with the driving speech, which has been widely explored recently. Existing works mostly neglect the person-specific talking style in generation, including facial expression and head pose styles. Several works intend to capture the personalities by fine-tuning modules. However, limited training data leads to the lack of vividness. In this work, we propose AdaMesh, a novel adaptive speech-driven facial animation approach, which learns the personalized talking style from a reference video of about 10 seconds and generates vivid facial expressions and head poses. Specifically, we propose mixture-of-low-rank adaptation (MoLoRA) to fine-tune the expression adapter, which efficiently captures the facial expression style. For the personalized pose style, we propose a pose adapter by building a discrete pose prior and retrieving the appropriate style embedding with a semantic-aware pose style matrix without fine-tuning. Extensive experimental results show that our approach outperforms state-of-the-art methods, preserves the talking style in the reference video, and generates vivid facial animation. The supplementary video and code will be available at https://adamesh.github.io.
Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model
Authors: Shiyuan Yang, Xiaodong Chen, Jing Liao
Date: 2023-10-11 00:00:00
Tasks: Denoising, Image Denoising, Image Generation, Image Inpainting

Recently, text-to-image denoising diffusion probabilistic models (DDPMs) have demonstrated impressive image generation capabilities and have also been successfully applied to image inpainting. However, in practice, users often require more control over the inpainting process beyond textual guidance, especially when they want to composite objects with customized appearance, color, shape, and layout. Unfortunately, existing diffusion-based inpainting methods are limited to single-modal guidance and require task-specific training, hindering their cross-modal scalability. To address these limitations, we propose Uni-paint, a unified framework for multimodal inpainting that offers various modes of guidance, including unconditional, text-driven, stroke-driven, exemplar-driven inpainting, as well as a combination of these modes. Furthermore, our Uni-paint is based on pretrained Stable Diffusion and does not require task-specific training on specific datasets, enabling few-shot generalizability to customized images. We have conducted extensive qualitative and quantitative evaluations that show our approach achieves comparable results to existing single-modal methods while offering multimodal inpainting capabilities not available in other methods. Code will be available at https://github.com/ysy31415/unipaint.
Multi-task Explainable Skin Lesion Classification
Authors: Mahapara Khurshid, Mayank Vatsa, Richa Singh
Date: 2023-10-11 00:00:00
Tasks: Classification, Lesion Classification, Segmentation, Skin Lesion Classification

Skin cancer is one of the deadliest diseases and has a high mortality rate if left untreated. The diagnosis generally starts with visual screening and is followed by a biopsy or histopathological examination. Early detection can aid in lowering mortality rates. Visual screening can be limited by the experience of the doctor. Due to the long tail distribution of dermatological datasets and significant intra-variability between classes, automatic classification utilizing computer-aided methods becomes challenging. In this work, we propose a multitask few-shot-based approach for skin lesions that generalizes well with few labelled data to address the small sample space challenge. The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network. The output of the segmentation network helps to focus on the most discriminatory features while making a decision by the classification network. To further enhance the classification performance, we have combined segmentation and classification loss in a weighted manner. We have also included the visualization results that explain the decisions made by the algorithm. Three dermatological datasets are used to evaluate the proposed method thoroughly. We also conducted cross-database experiments to ensure that the proposed approach is generalizable across similar datasets. Experimental results demonstrate the efficacy of the proposed work.
DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation
Authors: Rong Wang, Wei Mao, Hongdong Li
Date: 2023-10-11 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: 3D Pose Estimation, hand-object pose, Object, Pose Estimation

This paper addresses the task of 3D pose estimation for a hand interacting with an object from a single image observation. When modeling hand-object interaction, previous works mainly exploit proximity cues, while overlooking the dynamical nature that the hand must stably grasp the object to counteract gravity and thus preventing the object from slipping or falling. These works fail to leverage dynamical constraints in the estimation and consequently often produce unstable results. Meanwhile, refining unstable configurations with physics-based reasoning remains challenging, both by the complexity of contact dynamics and by the lack of effective and efficient physics inference in the data-driven learning framework. To address both issues, we present DeepSimHO: a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Specifically, for an initial hand-object pose estimated by a base network, we forward it to a physics simulator to evaluate its stability. However, due to non-smooth contact geometry and penetration, existing differentiable simulators can not provide reliable state gradient. To remedy this, we further introduce a deep network to learn the stability evaluation process from the simulator, while smoothly approximating its gradient and thus enabling effective back-propagation. Extensive experiments show that our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization. The code is available at https://github.com/rongakowang/DeepSimHO.
Exploiting Unfair Advantages: Investigating Opportunistic Trading in the NFT Market
Authors: Priyanka Bose, Dipanjan Das, Fabio Gritti, Nicola Ruaro, Christopher Kruegel, Giovanni Vigna
Date: 2023-09-05 00:00:00

As cryptocurrency evolved, new financial instruments, such as lending and borrowing protocols, currency exchanges, fungible and non-fungible tokens (NFT), staking and mining protocols have emerged. A financial ecosystem built on top of a blockchain is supposed to be fair and transparent for each participating actor. Yet, there are sophisticated actors who turn their domain knowledge and market inefficiencies to their strategic advantage; thus extracting value from trades not accessible to others. This situation is further exacerbated by the fact that blockchain-based markets and decentralized finance (DeFi) instruments are mostly unregulated. Though a large body of work has already studied the unfairness of different aspects of DeFi and cryptocurrency trading, the economic intricacies of non-fungible token (NFT) trades necessitate further analysis and academic scrutiny. The trading volume of NFTs has skyrocketed in recent years. A single NFT trade worth over a million US dollars, or marketplaces making billions in revenue is not uncommon nowadays. While previous research indicated the presence of wrongdoings in the NFT market, to our knowledge, we are the first to study predatory trading practices, what we call opportunistic trading, in depth. Opportunistic traders are sophisticated actors who employ automated, high-frequency NFT trading strategies, which, oftentimes, are malicious, deceptive, or, at the very least, unfair. Such attackers weaponize their advanced technical knowledge and superior understanding of DeFi protocols to disrupt trades of unsuspecting users, and collect profits from economic situations that are inaccessible to ordinary users, in a "supposedly" fair market. In this paper, we explore three such broad classes of opportunistic strategies aiming to realize three distinct trading objectives, viz., acquire, instant profit generation, and loss minimization.
Brave new world: Artificial Intelligence in teaching and learning
Authors: Adrian Groza, Anca Marginean
Date: 2023-09-27 00:00:00

We exemplify how Large Language Models are used in both teaching and learning. We also discuss the AI incidents that have already occurred in the education domain, and we argue for the urgent need to introduce AI policies in universities and for the ongoing strategies to regulate AI. Regarding policy for AI, our view is that each institution should have a policy for AI in teaching and learning. This is important from at least twofolds: (i) to raise awareness on the numerous educational tools that can both positively and negatively affect education; (ii) to minimise the risk of AI incidents in education.
BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer
Authors: Kunkun Pang, Dafei Qin, Yingruo Fan, Julian Habekost, Takaaki Shiratori, Junichi Yamagishi, Taku Komura
Date: 2023-09-07 00:00:00

Automatic gesture synthesis from speech is a topic that has attracted researchers for applications in remote communication, video games and Metaverse. Learning the mapping between speech and 3D full-body gestures is difficult due to the stochastic nature of the problem and the lack of a rich cross-modal dataset that is needed for training. In this paper, we propose a novel transformer-based framework for automatic 3D body gesture synthesis from speech. To learn the stochastic nature of the body gesture during speech, we propose a variational transformer to effectively model a probabilistic distribution over gestures, which can produce diverse gestures during inference. Furthermore, we introduce a mode positional embedding layer to capture the different motion speeds in different speaking modes. To cope with the scarcity of data, we design an intra-modal pre-training scheme that can learn the complex mapping between the speech and the 3D gesture from a limited amount of data. Our system is trained with either the Trinity speech-gesture dataset or the Talking With Hands 16.2M dataset. The results show that our system can produce more realistic, appropriate, and diverse body gestures compared to existing state-of-the-art approaches.
Automatic Control of Reactive Brain Computer Interfaces
Authors: Pex Tufvesson, Frida Heskebeck
Date: 2023-10-11 00:00:00
Tasks: Brain Computer Interface, Transfer Learning

This article discusses practical and theoretical aspects of real-time brain computer interface control methods based on Bayesian statistics. We investigate and improve the performance of automatic control and feedback algorithms of a reactive brain computer interface based on a visual oddball paradigm for faster statistical convergence. We introduce transfer learning using Gaussian mixture models, enabling a ready-to-use setup.
Deep Learning-Based Real-Time Rate Control for Live Streaming on Wireless Networks
Authors: Matin Mortaheb, Mohammad A. Amir Khojastepour, Srimat T. Chakradhar, Sennur Ulukus
Date: 2023-09-27 00:00:00

Providing wireless users with high-quality video content has become increasingly important. However, ensuring consistent video quality poses challenges due to variable encoded bitrate caused by dynamic video content and fluctuating channel bitrate caused by wireless fading effects. Suboptimal selection of encoder parameters can lead to video quality loss due to underutilized bandwidth or the introduction of video artifacts due to packet loss. To address this, a real-time deep learning based H.264 controller is proposed. This controller leverages instantaneous channel quality data driven from the physical layer, along with the video chunk, to dynamically estimate the optimal encoder parameters with a negligible delay in real-time. The objective is to maintain an encoded video bitrate slightly below the available channel bitrate. Experimental results, conducted on both QCIF dataset and a diverse selection of random videos from public datasets, validate the effectiveness of the approach. Remarkably, improvements of 10-20 dB in PSNR with repect to the state-of-the-art adaptive bitrate video streaming is achieved, with an average packet drop rate as low as 0.002.
Design of JiuTian Intelligent Network Simulation Platform
Authors: Lei Zhao, Miaomiao Zhang, Guangyu Li, Zhuowen Guan, Sijia Liu, Zhaobin Xiao, Yuting Cao, Zhe Lv, Yanping Liang
Date: 2023-09-28 00:00:00
Tasks: reinforcement-learning

This paper introduced the JiuTian Intelligent Network Simulation Platform, which can provide wireless communication simulation data services for the Open Innovation Platform. The platform contains a series of scalable simulator functionalities, offering open services that enable users to use reinforcement learning algorithms for model training and inference based on simulation environments and data. Additionally, it allows users to address optimization tasks in different scenarios by uploading and updating parameter configurations. The platform and its open services were primarily introduced from the perspectives of background, overall architecture, simulator, business scenarios, and future directions.
CAFA-evaluator: A Python Tool for Benchmarking Ontological Classification Methods
Authors: Damiano Piovesan, Davide Zago, Parnal Joshi, M. Clara De Paolis Kaluza, Mahta Mehdiabadi, Rashika Ramola, Alexander Miguel Monzon, Walter Reade, Iddo Friedberg, Predrag Radivojac, Silvio C. E. Tosatto
Date: 2023-10-10 00:00:00
Tasks: Benchmarking, Prediction

We present CAFA-evaluator, a powerful Python program designed to evaluate the performance of prediction methods on targets with hierarchical concept dependencies. It generalizes multi-label evaluation to modern ontologies where the prediction targets are drawn from a directed acyclic graph and achieves high efficiency by leveraging matrix computation and topological sorting. The program requirements include a small number of standard Python libraries, making CAFA-evaluator easy to maintain. The code replicates the Critical Assessment of protein Function Annotation (CAFA) benchmarking, which evaluates predictions of the consistent subgraphs in Gene Ontology. Owing to its reliability and accuracy, the organizers have selected CAFA-evaluator as the official CAFA evaluation software.
Hybrid System Stability Analysis of Multi-Lane Mixed-Autonomy Traffic
Authors: Sirui Li, Roy Dong, Cathy Wu
Date: 2023-10-11 00:00:00
Tasks: Autonomous Vehicles

Autonomous vehicles (AVs) hold vast potential to enhance transportation systems by reducing congestion, improving safety, and lowering emissions. AV controls lead to emergent traffic phenomena; one such intriguing phenomenon is traffic breaks (rolling roadblocks), where a single AV efficiently stabilizes multiple lanes through frequent lane switching, similar to the highway patrolling officers weaving across multiple lanes during difficult traffic conditions. While previous theoretical studies focus on single-lane mixed-autonomy systems, this work proposes a stability analysis framework for multi-lane systems under AV controls. Casting this problem into the hybrid system paradigm, the proposed analysis integrates continuous vehicle dynamics and discrete jumps from AV lane-switches. Through examining the influence of the lane-switch frequency on the system's stability, the analysis offers a principled explanation to the traffic break phenomena, and further discovers opportunities for less-intrusive traffic smoothing by employing less frequent lane-switching. The analysis further facilitates the design of traffic-aware AV lane-switch strategies to enhance system stability. Numerical analysis reveals a strong alignment between the theory and simulation, validating the effectiveness of the proposed stability framework in analyzing multi-lane mixed-autonomy traffic systems.
Super-twisting based sliding mode control of hydraulic actuator without velocity state
Authors: Manuel Estrada, Michael Ruderman, Leonid Fridman
Date: 2023-10-11 00:00:00

This paper provides a novel surface design and experimental evaluation of a super-twisting algorithm (STA) based control for hydraulic cylinder actuators. The proposed integral sliding surface allows to track a sufficiently smooth reference without using the velocity state which is hardly accessible in the noisy hydraulic systems. A design methodology based on LMI's is given, and the STA gains are designed to be adjusted by only one free parameter. The feasibility and effectiveness of the proposed control method are shown on a standard hydraulic test bench with one linear degree of freedom and passive load, where a typical motion profile is tracked with a bounded average error below 1 % from the total drive effective output.
Interaction-aware Traffic Prediction and Scenario-based Model Predictive Control for Autonomous Vehicles on Highways
Authors: Xiaorong Zhang, Sahar Zeinali, Georg Schildbach
Date: 2023-10-11 00:00:00
Tasks: Autonomous Vehicles, Decision Making, Model Predictive Control, Traffic Prediction

This paper addresses the problem of traffic prediction and control of autonomous vehicles on highways. A modified Interacting Multiple Model Kalman filter algorithm is applied to predict the motion behavior of the traffic participants by considering their interactions. A scenario generation component is used to produce plausible scenarios of the vehicles based on the predicted information. A novel integrated decision-making and control system is proposed by applying a Scenario-based Model Predictive Control approach. The designed controller considers safety, driving comfort, and traffic rules. The recursive feasibility of the controller is guaranteed under the inclusion of the `worst case' as an additional scenario to obtain safe inputs. Finally, the proposed scheme is evaluated using the HighD dataset. Simulation results indicate that the vehicle performs safe maneuvers in different traffic situations under the designed control framework.
Robust Adaptive Compensation of External Disturbances for Multi-Channel Linear Systems
Authors: V. H. Bui, A. A. Margun
Date: 2023-10-11 00:00:00

This paper proposes a new algorithm for compensating external disturbances for class of multi-channel linear systems. The solution to this problem is based on the use of the internal model principle and the extended error adaptation algorithm. It is assumed that the disturbance is the output of an autonomous linear generator with unknown parameters. At the first stage, a full-order observer with unknown input signals (Unknown Input Observer - UIO) is synthesized to solve the problem of estimating the state vector of this plant. Then a new observer of external disturbance is formed on the basis of state vector estimations. At the last stage, based on the new observer's estimations, a system with an extended state vector is formed for which a regulator providing compensation of disturbance is constructed. The performance of the obtained results is confirmed using computer simulation in MATLAB Simulink.