pwc-archive/papers-with-abstracts huggingface dataset:
PointClustering: Unsupervised Point Cloud Pre-Training Using Transformation Invariance in Clustering
Authors: Fuchen Long, Ting Yao, Zhaofan Qiu, Lusong Li, Tao Mei
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Clustering, Deep Clustering, Inductive Bias, Philosophy, Point Cloud Pre-training, Representation Learning

    Feature invariance under different data transformations, i.e., transformation invariance, can be regarded as a type of self-supervision for representation learning. In this paper, we present PointClustering, a new unsupervised representation learning scheme that leverages transformation invariance for point cloud pre-training. PointClustering formulates the pretext task as deep clustering and employs transformation invariance as an inductive bias, following the philosophy that common point cloud transformation will not change the geometric properties and semantics. Technically, PointClustering iteratively optimizes the feature clusters and backbone, and delves into the transformation invariance as learning regularization from two perspectives: point level and instance level. Point-level invariance learning maintains local geometric properties through gathering point features of one instance across transformations, while instance-level invariance learning further measures clusters over the entire dataset to explore semantics of instances. Our PointClustering is architecture-agnostic and readily applicable to MLP-based, CNN-based and Transformer-based backbones. We empirically demonstrate that the models pre-learnt on the ScanNet dataset by PointClustering provide superior performances on six benchmarks, across downstream tasks of classification and segmentation. More remarkably, PointClustering achieves an accuracy of 94.5% on ModelNet40 with Transformer backbone. Source code is available at https://github.com/FuchenUSTC/PointClustering.
Class Prototypes Based Contrastive Learning for Classifying Multi-Label and Fine-Grained Educational Videos
Authors: Rohit Gupta, Anirban Roy, Claire Christensen, Sujeong Kim, Sarah Gerard, Madeline Cincebeaux, Ajay Divakaran, Todd Grindal, Mubarak Shah
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Contrastive Learning, Math, Video Classification

    The recent growth in the consumption of online media by children during early childhood necessitates data-driven tools enabling educators to filter out appropriate educational content for young learners. This paper presents an approach for detecting educational content in online videos. We focus on two widely used educational content classes: literacy and math. For each class, we choose prominent codes (sub-classes) based on the Common Core Standards. For example, literacy codes include 'letter names', 'letter sounds', and math codes include 'counting', 'sorting'. We pose this as a fine-grained multilabel classification problem as videos can contain multiple types of educational content and the content classes can get visually similar (e.g., 'letter names' vs 'letter sounds'). We propose a novel class prototypes based supervised contrastive learning approach that can handle fine-grained samples associated with multiple labels. We learn a class prototype for each class and a loss function is employed to minimize the distances between a class prototype and the samples from the class. Similarly, distances between a class prototype and the samples from other classes are maximized. As the alignment between visual and audio cues are crucial for effective comprehension, we consider a multimodal transformer network to capture the interaction between visual and audio cues in videos while learning the embedding for videos. For evaluation, we present a dataset, APPROVE, employing educational videos from YouTube labeled with fine-grained education classes by education researchers. APPROVE consists of 193 hours of expert-annotated videos with 19 classes. The proposed approach outperforms strong baselines on APPROVE and other benchmarks such as Youtube-8M, and COIN. The dataset is available at https://nusci.csl.sri.com/project/APPROVE.
AstroNet: When Astrocyte Meets Artificial Neural Network
Authors: Mengqiao Han, Liyuan Pan, Xiabi Liu
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1

    Network structure learning aims to optimize network architectures and make them more efficient without compromising performance. In this paper, we first study the astrocytes, a new mechanism to regulate connections in the classic M-P neuron. Then, with the astrocytes, we propose an AstroNet that can adaptively optimize neuron connections and therefore achieves structure learning to achieve higher accuracy and efficiency. AstroNet is based on our built Astrocyte-Neuron model, with a temporal regulation mechanism and a global connection mechanism, which is inspired by the bidirectional communication property of astrocytes. With the model, the proposed AstroNet uses a neural network (NN) for performing tasks, and an astrocyte network (AN) to continuously optimize the connections of NN, i.e., assigning weight to the neuron units in the NN adaptively. Experiments on the classification task demonstrate that our AstroNet can efficiently optimize the network structure while achieving state-of-the-art (SOTA) accuracy.
Data-Free Knowledge Distillation via Feature Exchange and Activation Region Constraint
Authors: Shikang Yu, Jiachen Chen, Hu Han, Shuqiang Jiang
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Data Augmentation, Data-free Knowledge Distillation, Knowledge Distillation, Synthetic Data Generation

    Despite the tremendous progress on data-free knowledge distillation (DFKD) based on synthetic data generation, there are still limitations in diverse and efficient data synthesis. It is naive to expect that a simple combination of generative network-based data synthesis and data augmentation will solve these issues. Therefore, this paper proposes a novel data-free knowledge distillation method (SpaceshipNet) based on channel-wise feature exchange (CFE) and multi-scale spatial activation region consistency (mSARC) constraint. Specifically, CFE allows our generative network to better sample from the feature space and efficiently synthesize diverse images for learning the student network. However, using CFE alone can severely amplify the unwanted noises in the synthesized images, which may result in failure to improve distillation learning and even have negative effects. Therefore, we propose mSARC to assure the student network can imitate not only the logit output but also the spatial activation region of the teacher network in order to alleviate the influence of unwanted noises in diverse synthetic images on distillation learning. Extensive experiments on CIFAR-10, CIFAR-100, Tiny-ImageNet, Imagenette, and ImageNet100 show that our method can work well with different backbone networks, and outperform the state-of-the-art DFKD methods. Code will be available at: https://github.com/skgyu/SpaceshipNet.
WINNER: Weakly-Supervised hIerarchical decompositioN and aligNment for Spatio-tEmporal Video gRounding
Authors: Mengze Li, Han Wang, Wenqiao Zhang, Jiaxu Miao, Zhou Zhao, Shengyu Zhang, Wei Ji, Fei Wu
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Contrastive Learning, Spatio-Temporal Video Grounding, Video Grounding

Spatio-temporal video grounding aims to localize the aligned visual tube corresponding to a language query. Existing techniques achieve such alignment by exploiting dense boundary and bounding box annotations, which can be prohibitively expensive. To bridge the gap, we investigate the weakly-supervised setting, where models learn from easily accessible video-language data without annotations. We identify that intra-sample spurious correlations among video-language components can be alleviated if the model captures the decomposed structures of video and language data. In this light, we propose a novel framework, namely WINNER, for hierarchical video-text understanding. WINNER first builds the language decomposition tree in a bottom-up manner, upon which the structural attention mechanism and top-down feature backtracking jointly build a multi-modal decomposition tree, permitting a hierarchical understanding of unstructured videos. The multi-modal decomposition tree serves as the basis for multi-hierarchy language-tube matching. A hierarchical contrastive learning objective is proposed to learn the multi-hierarchy correspondence and distinguishment with intra-sample and inter-sample video-text decomposition structures, achieving video-language decomposition structure alignment. Extensive experiments demonstrate the rationality of our design and its effectiveness beyond state-of-the-art weakly supervised methods, even some supervised methods.
An Actor-Centric Causality Graph for Asynchronous Temporal Inference in Group Activity
Authors: Zhao Xie, Tian Gao, Kewei Wu, Jiao Chang
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Activity Recognition, Group Activity Recognition, regression, Relation

    The causality relation modeling remains a challenging task for group activity recognition. The causality relations describe the influence of some actors (cause actors) on other actors (effect actors). Most existing graph models focus on learning the actor relation with synchronous temporal features, which is insufficient to deal with the causality relation with asynchronous temporal features. In this paper, we propose an Actor-Centric Causality Graph Model, which learns the asynchronous temporal causality relation with three modules, i.e., an asynchronous temporal causality relation detection module, a causality feature fusion module, and a causality relation graph inference module. First, given a centric actor and correlative actor, we analyze their influences to detect causality relation. We estimate the self influence of the centric actor with self regression. We estimate the correlative influence from the correlative actor to the centric actor with correlative regression, which uses asynchronous features at different timestamps. Second, we synchronize the two action features by estimating the temporal delay between the cause action and the effect action. The synchronized features are used to enhance the feature of the effect action with a channel-wise fusion. Third, we describe the nodes (actors) with causality features and learn the edges by fusing the causality relation with the appearance relation and distance relation. The causality relation graph inference provides crucial features of effect action, which are complementary to the base model using synchronous relation inference. The two relation inferences are aggregated to enhance group relation learning. Extensive experiments show that our method achieves state-of-the-art performance on the Volleyball dataset and Collective Activity dataset.
Learning a Simple Low-Light Image Enhancer From Paired Low-Light Instances
Authors: Zhenqi Fu, Yan Yang, Xiaotong Tu, Yue Huang, Xinghao Ding, Kai-Kuang Ma
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Image Enhancement, Low-Light Image Enhancement

    Low-light Image Enhancement (LIE) aims at improving contrast and restoring details for images captured in low-light conditions. Most of the previous LIE algorithms adjust illumination using a single input image with several handcrafted priors. Those solutions, however, often fail in revealing image details due to the limited information in a single image and the poor adaptability of handcrafted priors. To this end, we propose PairLIE, an unsupervised approach that learns adaptive priors from low-light image pairs. First, the network is expected to generate the same clean images as the two inputs share the same image content. To achieve this, we impose the network with the Retinex theory and make the two reflectance components consistent. Second, to assist the Retinex decomposition, we propose to remove inappropriate features in the raw image with a simple self-supervised mechanism. Extensive experiments on public datasets show that the proposed PairLIE achieves comparable performance against the state-of-the-art approaches with a simpler network and fewer handcrafted priors. Code is available at: https://github.com/zhenqifu/PairLIE.
A Characteristic Function-Based Method for Bottom-Up Human Pose Estimation
Authors: Haoxuan Qu, Yujun Cai, Lin Geng Foo, Ajay Kumar, Jun Liu
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Pose Estimation

    Most recent methods formulate the task of human pose estimation as a heatmap estimation problem, and use the overall L2 loss computed from the entire heatmap to optimize the heatmap prediction. In this paper, we show that in bottom-up human pose estimation where each heatmap often contains multiple body joints, using the overall L2 loss to optimize the heatmap prediction may not be the optimal choice. This is because, minimizing the overall L2 loss cannot always lead the model to locate all the body joints across different sub-regions of the heatmap more accurately. To cope with this problem, from a novel perspective, we propose a new bottom-up human pose estimation method that optimizes the heatmap prediction via minimizing the distance between two characteristic functions respectively constructed from the predicted heatmap and the groundtruth heatmap. Our analysis presented in this paper indicates that the distance between these two characteristic functions is essentially the upper bound of the L2 losses w.r.t. sub-regions of the predicted heatmap. Therefore, via minimizing the distance between the two characteristic functions, we can optimize the model to provide a more accurate localization result for the body joints in different sub-regions of the predicted heatmap. We show the effectiveness of our proposed method through extensive experiments on the COCO dataset and the CrowdPose dataset.
Learning To Segment Every Referring Object Point by Point
Authors: Mengxue Qu, Yu Wu, Yunchao Wei, Wu Liu, Xiaodan Liang, Yao Zhao
Date: 2023-01-01 00:00:00
Proceeding: CVPR 2023 1
Tasks: Object, Referring Expression, Referring Expression Segmentation

    Referring Expression Segmentation (RES) can facilitate pixel-level semantic alignment between vision and language. Most of the existing RES approaches require massive pixel-level annotations, which are expensive and exhaustive. In this paper, we propose a new partially supervised training paradigm for RES, i.e., training using abundant referring bounding boxes and only a few (e.g., 1%) pixel-level referring masks. To maximize the transferability from the REC model, we construct our model based on the point-based sequence prediction model. We propose the co-content teacher-forcing to make the model explicitly associate the point coordinates (scale values) with the referred spatial features, which alleviates the exposure bias caused by the limited segmentation masks. To make the most of referring bounding box annotations, we further propose the resampling pseudo points strategy to select more accurate pseudo-points as supervision. Extensive experiments show that our model achieves 52.06% in terms of accuracy (versus 58.93% in fully supervised setting) on RefCOCO+@testA, when only using 1% of the mask annotations.
A HYBRID METHOD FOR SURROGATE MODEL UPDATING IN ENGINEERING DESIGN OPTIMIZATION
Authors: Tiefu Shao, Sundar Krishnamurty
Date: 2007-09-04 00:00:00
Proceeding: Proceedings of the ASME 2007 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference 2007 9

This paper addresses the critical issue of effectiveness, efficiency, and reliability in simulation-based design optimization under surrogate model uncertainty. Specifically, it presents a novel method to build surrogate models iteratively with sufficient fidelity for accurately capturing global optimal design solutions at a minimal cost. The salient feature of the
proposed method lies in its unique preference of focusing necessarily high fidelity at potential global optimal regions of
surrogate models. The proposed method is the synergic integration of the multiple preference point method, which
updates surrogate model at current local optimal points predicted with data-mining techniques in genetic algorithm
setup, and the maximum variance point method, which updates surrogate model at the point associated with the maximum
prediction variance. Through illustrative comparison studies on thirty different optimization scenarios derived from 15 different test functions, the proposed method demonstrates the tangible reliability advancement. The experimental results indicate that the proposed method can be a reliable updating method in surrogate-model-based design optimization for efficiently locating the global optimal point/points in various kinds of optimization scenarios featured by single/multiple global optimal point/points that may exist at the corners of design space, inside design space, or on the boundaries of design space
Adaptive sequential sampling for surrogate model generation with artificial neural networks
Authors: John Eason, Selen Cremaschi
Date: 2014-06-07 00:00:00
Proceeding: Computers and Chemical Engineering 2014 6

Surrogate models – simple functional approximations of complex models – can facilitate engineering
analysis of complicated systems by greatly reducing computational expense. The construction of a surrogate model requires evaluation of the original model to gather the data necessary for building the
surrogate. Sequential sampling procedures are proposed for determining and minimizing the required
number of samples for efficient global surrogate construction. In this paper, two new adaptive sampling algorithms – one purely adaptive and one combining adaptive and space-filling characteristics –
are proposed and compared to a purely space-filling approach. Our analysis suggests a mixed adaptive
sampling approach for constructing surrogates for systems where the behavior of the underlying model
is unknown. Results of the case study, optimization of carbon dioxide capture process with aqueous
amines, revealed that the mixed adaptive sampling algorithm may reduce the required sample size by
up to 40% compared to a purely space-filling design
Classifying Clustering Schemes
Authors: Gunnar Carlsson, Facundo Memoli
Date: 2010-11-24 00:00:00
Tasks: Clustering

Many clustering schemes are defined by optimizing an objective function defined on the partitions of the underlying set of a finite metric space. In this paper, we construct a framework for studying what happens when we instead impose various structural conditions on the clustering schemes, under the general heading of functoriality. Functoriality refers to the idea that one should be able to compare the results of clustering algorithms as one varies the data set, for example by adding points or by applying functions to it. We show that within this framework, one can prove a theorems analogous to one of J. Kleinberg, in which for example one obtains an existence and uniqueness theorem instead of a non-existence result. We obtain a full classification of all clustering schemes satisfying a condition we refer to as excisiveness. The classification can be changed by varying the notion of maps of finite metric spaces. The conditions occur naturally when one considers clustering as the statistical version of the geometric notion of connected components. By varying the degree of functoriality that one requires from the schemes it is possible to construct richer families of clustering schemes that exhibit sensitivity to density.
PCCA-Model: an attention module for medical image segmentation
Authors: LINJIE LIU, Guanglei Wang, * YANLIN WU, Hongrui Wang, AND YAN LI
Date: 2023-04-01 00:00:00
Proceeding: Biomedical Optics Express 2023 4
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Convolutional neural networks have been increasingly employed in the field of
medical image segmentation. Based on the idea that the human visual cortex differs in terms
of the size of the receptive field and can sense the stimulus location, we propose the pyramid
channel coordinate attention (PCCA) module to fuse multiscale features in the channel direction,
aggregate local and global channel information, combine them with the location information in
the spatial direction, and then integrate them into the existing semantic segmentation network.
We conducted numerous experiments on the datasets, namely LiTS, ISIC-2018, and CX, and
obtained state-of-the-art results.
DMCNN: A Deep Multiscale Convolutional Neural Network Model for Medical Image Segmentation
Authors: Lin Teng, 1Hang Li, 1and Shahid Karim
Date: 2019-12-27 00:00:00
Proceeding: Journal of Healthcare Engineering 2019 12
Tasks: Data Augmentation, Decoder, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Medical image segmentation is one of the hot issues in the related area of image processing. Precise segmentation for medical images is a vital guarantee for follow-up treatment. At present, however, low gray contrast and blurred tissue boundaries are common in medical images, and the segmentation accuracy of medical images cannot be effectively improved. Especially, deep learning methods need more training samples, which lead to time-consuming process. Therefore, we propose a novelty model for medical image segmentation based on deep multiscale convolutional neural network (CNN) in this article. First, we extract the region of interest from the raw medical images. Then, data augmentation is operated to acquire more training datasets. Our proposed method contains three models: encoder, U-net, and decoder. Encoder is mainly responsible for feature extraction of 2D image slice. The U-net cascades the features of each block of the encoder with those obtained by deconvolution in the decoder under different scales. The decoding is mainly responsible for the upsampling of the feature graph after feature extraction of each group. Simulation results show that the new method can boost the segmentation accuracy. And, it has strong robustness compared with other segmentation methods.
Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl
Authors: Juan C. Caicedo, Allen Goodman, Kyle W. Karhohs, Beth A. Cimini, Jeanelle Ackerman, Marzieh Haghighi, CherKeng Heng, Tim Becker, Minh Doan, Claire McQuin, Mohammad Rohban, Shantanu Singh & Anne E. Carpenter
Date: 2020-01-22 00:00:00
Proceeding: 自然方法 2020 1
Tasks: Segmentation

Segmenting the nuclei of cells in microscopy images is often the first step in the quantitative analysis of imaging data for biological and biomedical applications. Many bioimage analysis tools can segment nuclei in images but need to be selected and configured for every experiment. The 2018 Data Science Bowl attracted 3,891 teams worldwide to make the first attempt to build a segmentation method that could be applied to any two-dimensional light microscopy image of stained nuclei across experiments, with no human interaction. Top participants in the challenge succeeded in this task, developing deep-learning-based models that identified cell nuclei across many image types and experimental conditions without the need to manually adjust segmentation parameters. This represents an important step toward configuration-free bioimage analysis software tools.