pwc-archive/papers-with-abstracts huggingface dataset:
Video-based marker-free tracking and multi-scale analysis of mouse locomotor activity and behavioral aspects in an open field arena: A perspective approach to the quantification of complex gait disturbances associated with Alzheimer's disease
Authors: Bogachev Mikhail, Sinitca Aleksandr, Grigarevichius Konstantin, Pyko Nikita, Lyanova Asya, Tsygankova Margarita, Davletshin Eldar, Petrov Konstantin, Ageeva Tatyana, Pyko Svetlana, Kaplun Dmitrii, Kayumov Airat, Mukhamedshina Yana
Date: 2023-02-02 00:00:00
Proceeding: Frontiers in Neuroinformatics 2023 2
Tasks: Trajectory Modeling

Introduction: Complex gait disturbances represent one of the prominent manifestations of various neurophysiological conditions, including widespread neurodegenerative disorders such as Alzheimer's and Parkinson's diseases. Therefore, instrumental measurement techniques and automatic computerized analysis appears essential for the differential diagnostics, as well as for the assessment of treatment effectiveness from experimental animal models to clinical settings.

Methods: Here we present a marker-free instrumental approach to the analysis of gait disturbances in animal models. Our approach is based on the analysis of video recordings obtained with a camera placed underneath an open field arena with transparent floor using the DeeperCut algorithm capable of online tracking of individual animal body parts, such as the snout, the paws and the tail. The extracted trajectories of animal body parts are next analyzed using an original computerized methodology that relies upon a generalized scalable model based on fractional Brownian motion with parameters identified by detrended partial cross-correlation analysis.

Results: We have shown that in a mouse model representative movement patterns are characterized by two asymptotic regimes characterized by integrated 1/f noise at small scales and nearly random displacements at large scales separated by a single crossover. More detailed analysis of gait disturbances revealed that the detrended cross-correlations between the movements of the snout, paws and tail relative to the animal body midpoint exhibit statistically significant discrepancies in the Alzheimer's disease mouse model compared to the control group at scales around the location of the crossover.

Discussion: We expect that the proposed approach, due to its universality, robustness and clear physical interpretation, is a promising direction for the design of applied analysis tools for the diagnostics of various gait disturbances and behavioral aspects in animal models. We further believe that the suggested mathematical models could be relevant as a complementary tool in clinical diagnostics of various neurophysiological conditions associated with movement disorders.
Impact of Image Compression on In Vitro Cell Migration Analysis
Authors: Ehsaneddin Jalilian, Michael Linortner, Andreas Uhl.
Date: 2023-05-04 00:00:00
Proceeding: Computers 2023 5
Tasks: Cell Segmentation, Image Compression, MS-SSIM, Segmentation, SSIM

Collective cell movement is an indication of phenomena such as wound healing, embryonic morphogenesis, cancer invasion, and metastasis. Wound healing is a complicated cellular and biochemical procedure in which skin cells migrate from the wound boundaries into the wound area to reconstruct the injured skin layer(s). In vitro analysis of cell migration is an effective assay for measuring changes in cell migratory complement in response to experimental inspections. Open-source segmentation software (e.g., an ImageJ plug-in) is available to analyze images of in vitro scratch wound healing assays; however, often, these tools are error-prone when applied to, e.g., low-contrast, out-of-focus, and noisy images, and require manual tuning of various parameters, which is imprecise, tedious, and time-consuming. We propose two algorithmic methods (namely log gradient segmentation and entropy filter segmentation) for cell segmentation and the subsequent measurement of the collective cell migration in the corresponding microscopic imagery. We further investigate the effects of image compression on the algorithms’ measurement accuracy, applying lossy compression algorithms (the current ISO standards JPEG2000, JPEG, JPEG-XL and AV1, BPG, and WEBP). We aim to identify the most suitable compression algorithm that can be used for this purpose, relating rate–distortion performance as measured in terms of peak signal-to-noise ratio (PSNR) and the multiscale structural similarity index (MS-SSIM) to the segmentation accuracy obtained by the segmentation algorithms. The experimental results show that the log gradient segmentation algorithm provides robust performance for segmenting the wound area, whereas the entropy filter segmentation algorithm is unstable for this purpose under certain circumstances. Additionally, the best-suited compression strategy is observed to be dependent on (i) the segmentation algorithm used and (ii) the actual data sequence being processed.
Co-Attention Aligned Mutual Cross-Attention for Cloth-Changing Person Re-Identification
Authors: Qizao Wang, Xuelin Qian, Yanwei Fu, xiangyang xue
Date: 2023-03-08 00:00:00
Proceeding: Asian Conference on Computer Vision (ACCV) 2023 3
Tasks: Cloth-Changing Person Re-Identification, Person Re-Identification, Person Retrieval, Retrieval

Person re-identification (Re-ID) has been widely studied and achieved significant progress. However, traditional person Re-ID methods primarily rely on cloth-related color appearance, which is unreliable under real-world scenarios when people change their clothes. Cloth-changing person Re-ID that takes this problem into account has received increasing attention recently, but it is more challenging to learn discriminative person identity features, since larger intra-class variation and smaller inter-class easily occur in the image feature space with clothing changes. Beyond appearance features, some known identity-related features can be implicitly encoded in images (\textit{e.g.}, body shapes). In this paper, we first design a novel Shape Semantics Embedding (SSE) module to encode body shape semantic information, which is one of the essential clues to distinguish pedestrians when their clothes change. To better complement image features, we further propose a Co-attention Aligned Mutual Cross-attention (CAMC) framework. Different from previous attention-based fusion strategies, it first aligns features from multiple modalities, then effectively interacts and transfers identity-aware but cloth-irrelevant knowledge between the image space and the body shape space, resulting in a more robust feature representation. To the best of our knowledge, this is the first work to adopt Transformer to handle the multi-modal interaction for cloth-changing person Re-ID. Extensive experiments demonstrate the effectiveness of our proposed method and show the superior performance achieved on several cloth-changing person Re-ID benchmarks. Codes will be available at \url{https://github.com/QizaoWang/CAMC-CCReID}.
A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition
Authors: Saining Zhang, Yuhang Zhang, Ye Zhang, YuFei Wang, Zhigang Song
Date: 2023-08-25 00:00:00
Proceeding: journal 2023 8
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER), Image Classification

In recent years, facial expression recognition (FER) has garnered significant attention within the realm of computer vision research. This paper presents an innovative network called the Dual-Direction Attention Mixed Feature Network (DDAMFN) specifically designed for FER, boasting both robustness and lightweight characteristics. The network architecture comprises two primary components: the Mixed Feature Network (MFN) serving as the backbone, and the Dual-Direction Attention Network (DDAN) functioning as the head. To enhance the network’s capability in the MFN, resilient features are extracted by utilizing mixed-size kernels. Additionally, a new Dual-Direction Attention (DDA) head that generates attention maps in two orientations is proposed, enabling the model to capture long-range dependencies effectively. To further improve the accuracy, a novel attention loss mechanism for the DDAN is introduced with different heads focusing on distinct areas of the input. Experimental evaluations on several widely used public datasets, including AffectNet, RAF-DB, and FERPlus, demonstrate the superiority of the DDAMFN compared to other existing models, which establishes that the DDAMFN as the state-of-the-art model in the field of FER.
Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Plans
Authors: Kyowoon Lee, Seongun Kim, Jaesik Choi
Date: 2023-10-30 00:00:00
Proceeding: NeurIPS 2023 11

Diffusion-based planning has shown promising results in long-horizon, sparse-reward tasks by training trajectory diffusion models and conditioning the sampled trajectories using auxiliary guidance functions. However, due to their nature as generative models, diffusion models are not guaranteed to generate feasible plans, resulting in failed execution and precluding planners from being useful in safety-critical applications. In this work, we propose a novel approach to refine unreliable plans generated by diffusion models by providing refining guidance to error-prone plans. To this end, we suggest a new metric named restoration gap for evaluating the quality of individual plans generated by the diffusion model. A restoration gap is estimated by a gap predictor which produces restoration gap guidance to refine a diffusion planner. We additionally present an attribution map regularizer to prevent adversarial refining guidance that could be generated from the sub-optimal gap predictor, which enables further refinement of infeasible plans. We demonstrate the effectiveness of our approach on three different benchmarks in offline control settings that require long-horizon planning. We also illustrate that our approach presents explainability by presenting the attribution maps of the gap predictor and highlighting error-prone transitions, allowing for a deeper understanding of the generated plans.
Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All?
Authors: Haitao Mao, Zhikai Chen, Wei Jin, Haoyu Han, Yao Ma, Tong Zhao, Neil Shah, Jiliang Tang
Date: 2023-06-02 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: All, Node Classification

Recent studies on Graph Neural Networks(GNNs) provide both empirical and theoretical evidence supporting their effectiveness in capturing structural patterns on both homophilic and certain heterophilic graphs. Notably, most real-world homophilic and heterophilic graphs are comprised of a mixture of nodes in both homophilic and heterophilic structural patterns, exhibiting a structural disparity. However, the analysis of GNN performance with respect to nodes exhibiting different structural patterns, e.g., homophilic nodes in heterophilic graphs, remains rather limited. In the present study, we provide evidence that Graph Neural Networks(GNNs) on node classification typically perform admirably on homophilic nodes within homophilic graphs and heterophilic nodes within heterophilic graphs while struggling on the opposite node set, exhibiting a performance disparity. We theoretically and empirically identify effects of GNNs on testing nodes exhibiting distinct structural patterns. We then propose a rigorous, non-i.i.d PAC-Bayesian generalization bound for GNNs, revealing reasons for the performance disparity, namely the aggregated feature distance and homophily ratio difference between training and testing nodes. Furthermore, we demonstrate the practical implications of our new findings via (1) elucidating the effectiveness of deeper GNNs; and (2) revealing an over-looked distribution shift factor on graph out-of-distribution problem and proposing a new scenario accordingly.
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition
Authors: Shuhuai Ren, Aston Zhang, Yi Zhu, Shuai Zhang, Shuai Zheng, Mu Li, Alex Smola, Xu sun
Date: 2023-04-10 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: image-classification, Image Classification, object-detection, Object Detection, Segmentation, Semantic Segmentation

This work proposes POMP, a prompt pre-training method for vision-language models. Being memory and computation efficient, POMP enables the learned prompt to condense semantic information for a rich set of visual concepts with over twenty-thousand classes. Once pre-trained, the prompt with a strong transferable ability can be directly plugged into a variety of visual recognition tasks including image classification, semantic segmentation, and object detection, to boost recognition performances in a zero-shot manner. Empirical evaluation shows that POMP achieves state-of-the-art performances on 21 datasets, e.g., 67.0% average accuracy on 10 classification datasets (+3.1% compared to CoOp) and 84.4 hIoU on open-vocabulary Pascal VOC segmentation (+6.9 compared to ZSSeg). Our code is available at https://github.com/amazon-science/prompt-pretraining.
RFM: response-aware feedback mechanism for background based conversation
Authors: Jiatao Chen, Biqing Zeng, Zhibin Du, Huimin Deng, Mayi Xu, Zibang Gan & Meirong Ding
Date: 2022-08-26 00:00:00
Proceeding: journal 2022 8
Tasks: Response Generation

In general, the existing dialogue systems tend to generate generic responses due to lack of external knowledge. One of the usual solutions is the Background Based Conversations (BBCs), which can help dialogue systems generate more informative and appropriate responses, based on an external knowledge source. Unfortunately, there still exists some difficulties for BBCs when correcting the selected knowledge during response generation, e.g., see GTTP, CaKe. In this paper, we propose a novel architecture called Response-aware Feedback Mechanism (RFM) for BBCs to address this shortcoming. The main advantage is that a Response-aware Feedback Weight Vector is introduced to integrate the background knowledge and responses, so that the knowledge selector could select more accurate knowledge. With the help of this self-correcting mechanism, the selected knowledge is adjusted and corrected dynamically in each decoding time step. As an application, we carry out experiments on the Holl-E and Wizard of Wikipedia datasets, the results indicate that the RFM model has much better performance on automatic and human evaluation, compare to eleven state-of-the-art methods, including RefNet and GLKS.
Improving Span-based Aspect Sentiment Triplet Extraction with Abundant Syntax Knowledge
Authors: Lingcong Feng, Biqing Zeng, Lewei He, Mayi Xu, Huimin Deng, Pengfei Chen, Zipeng Huang & Weihua Du
Date: 2022-12-23 00:00:00
Proceeding: journal 2022 12
Tasks: Aspect-Based Sentiment Analysis, Aspect Sentiment Triplet Extraction, Sentiment Analysis, Triplet

Aspect sentiment triplet extraction is a subtask of aspect based sentiment analysis, which has attracted considerable attention in recent years. It aims to obtain the aspect term, the associated opinion term and sentiment polarity, so called the triplet in reviews. Most existing studies can only learn the word-level context information and capture the superficial interactions between aspect and opinion terms. In contrast, our proposed span-based aspect sentiment triplet extraction model incorporates abundant syntax information, namely the span-level part-of-speech, constituent syntax, and dependency syntax information. Therefore, our model can learn both the connections between multiple words within the span and the interactions between aspect and opinion terms, and thus improve the performance on triplet extraction. Extensive experimental results show that our model outperforms previous models on four benchmark datasets and achieves state-of-the-art performance.
Combining dynamic local context focus and dependency cluster attention for aspect-level sentiment classification
Authors: Mayi Xu, Biqing Zeng, Heng Yang, Junlong Chi, Jiatao Chen, Hongye Liu a
Date: 2022-03-14 00:00:00
Proceeding: journal 2022 3
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Sentence, Sentiment Analysis, Sentiment Classification

Aspect-level sentiment classification (ASC), as a subtask of Aspect-based sentiment analysis (ABSA), aims to analyze the sentiment polarity of different aspect terms in a sentence. Although the existing methods have been proven to be effective, they fail to effectively identify the range of local context and fully leverage the nonequivalent property of dependency relation. Hence, we propose a concept of dependency cluster and design two modules named Dynamic Local Context Focus (DLCF) and Dependency Cluster Attention (DCA) respectively. The DLCF can dynamically capture the range of local context based on the different max distance from the target aspect term to its context words and the DCA allows the model to pay more attention to the cluster which is more critical for sentiment classification. Together with the two modules, we then propose the DLCF-DCA model, in which the DLCF is equipped before DCA. Considering the DLCF has masked or weighted down the less-semantic-relative words, the semantic information can therefore be better extracted in DCA. Experiments conducted on six benchmark datasets demonstrate that DLCF-DCA achieves the state-of-the-art results. Moreover, the ablation experiment results also verify the effectiveness of each part in DLCF-DCA.
Deep Image Feature Learning With Fuzzy Rules
Authors: Xiang Ma; Liangzhe Chen; Zhaohong Deng; Peng Xu; Qisheng Yan; Kup-Sze Choi; Shitong Wang
Date: 2023-04-03 00:00:00
Proceeding: journal 2023 4

Feature extraction methods are key to many image processing tasks. At present, the most popular method is to use a deep neural network to automatically extract features through end-to-end training instead of the traditional hand-crafted feature extraction. However, the training of deep neural network relies heavily on data quality and quantity, and the network is a black-box model that has poor interpretability. Human intelligence can be leveraged here to improve the deep neural network model, where the human decision process can be integrated in feature learning and object classification to enhance robustness and interpretability. In this paper, the method Deep Image Feature Learning with Fuzzy Rules (DIFL-FR) is proposed, where human decision process is embedded in feature extraction by combining fuzzy logic rule-based modeling with deep-stacked learning strategy. The proposed method has the following distinctive characteristics. First, since the method is based on fuzzy sets and fuzzy inference, it can extract more robust features from noisy data scenes. Second, the method progressively learns the image features through a layer-by-layer approach based on fuzzy rules, so that the feature learning process can be better explained by the rules generated. Third, the learning process of the method has a high efficiency since it is only based on forward propagation without back propagation and iterative learning. Finally, while the method is based on unsupervised learning, it can be easily extended to supervised and semi-supervised learning cases. The results of extensive experiments conducted on image datasets of different scales clearly show the effectiveness of the proposed method.
One-Step Multiview Fuzzy Clustering With Collaborative Learning Between Common and Specific Hidden Space Information
Authors: Wei Zhang; Zhaohong Deng; Te Zhang; Kup-Sze Choi; Shitong Wang
Date: 2023-05-22 00:00:00
Proceeding: journal 2023 5
Tasks: Clustering, Multiview Clustering

Multiview data are widespread in real-world applications, and multiview clustering is a commonly used technique to effectively mine the data. Most of the existing algorithms perform multiview clustering by mining the commonly hidden space between views. Although this strategy is effective, there are two challenges that still need to be addressed to further improve the performance. First, how to design an efficient hidden space learning method so that the learned hidden spaces contain both shared and specific information of multiview data. Second, how to design an efficient mechanism to make the learned hidden space more suitable for the clustering task. In this study, a novel one-step multiview fuzzy clustering (OMFC-CS) method is proposed to address the two challenges by collaborative learning between the common and specific space information. To tackle the first challenge, we propose a mechanism to extract the common and specific information simultaneously based on matrix factorization. For the second challenge, we design a one-step learning framework to integrate the learning of common and specific spaces and the learning of fuzzy partitions. The integration is achieved in the framework by performing the two learning processes alternately and thereby yielding mutual benefit. Furthermore, the Shannon entropy strategy is introduced to obtain the optimal views weight assignment during clustering. The experimental results based on benchmark multiview datasets demonstrate that the proposed OMFC-CS outperforms many existing methods.
Self-organizing interval type-2 fuzzy neural network with adaptive discriminative strategy
Authors: Honggui Han; Chenxuan Sun; Xiaolong Wu  ; Hongyan Yang; Junfei Qiao
Date: 2022-10-25 00:00:00
Proceeding: journal 2022 10

Covariate shift is a critical issue of interval type-2 fuzzy neural networks (IT2FNNs) due to the distribution discrepancy between training and testing samples. In this situation, IT2FNNs usually struggle to identify potential features from samples with explicit inductive biases. To address this problem, a self-organizing IT2FNN with an adaptive discriminative strategy (ADS-SOIT2FNN) is developed to maintain the identification performance in the presence of covariate shift. First, a granularity-based metric (GM), using higher order statistics of local samples, is designed to distinguish the distribution discrepancy caused by covariate shift. The multiple kernels incorporated into GM are able to cover the sample features of the whole Hilbert space. Second, a self-organizing strategy, associated with GM-based discriminative information, is presented to alleviate the structural bias by growing and pruning fuzzy rules. Then, a compact structure of ADS-SOIT2FNN is achieved to adapt to the covariate shift of samples and further strengthen its inductive ability. Third, an adaptive risk mitigation learning algorithm (RMLA) is introduced to update the parameters of ADS-SOIT2FNN. RMLA can regulate the derivatives of parameters with arbitrary distribution samples, which is beneficial for maintaining the global accuracy by relieving the risk of parameter biases. Finally, the effectiveness of ADS-SOIT2FNN is verified by some experiments for identifying nonlinear systems with covariate shift.