Reviewerly/exHarmony huggingface dataset:
MS UX-Net: A Multi-scale Depth-Wise Convolution Network for Medical Image Segmentation

Semantic segmentation of 3D medical images plays an important role in assisting physicians in diagnosing and successively studying the progression of the disease. In recent years, transformer-based models have achieved state-of-the-art performances on several 3D medical image segmentation tasks. However, these methods still suffer from huge model sizes and high complexity. On the other hand, large-kernel depth-wise convolution networks have shown great potential to encode contextual information more efficiently and effectively than transformer-based networks in natural image segmentation tasks. Inspired by the success of large-kernel depth-wise convolution networks and their variants, we propose a multi-scale lightweight depth-wise convolution network termed MS UX-Net, a U-shaped network mainly composed of convolution operations with different kernel sizes. Specifically, we design a multi-scale feature extraction module for feature encoding, which extracts features into four different scales and learns special features of different scales effectively. Furthermore, we adopt multi-scale depth-wise convolution rather than single-scale standard convolution during the decoding stages, which yields a notable reduction in both the number of model parameters and the computational complexity. The competitive results on two public FLARE2021 and Synapse datasets and a private Pancreatic tumor dataset demonstrate the effectiveness of our method.
A Pre-trained Model for Chinese Medical Record Punctuation Restoration

In the medical field, text after automatic speech recognition (ASR) is poorly readable due to a lack of punctuation. Even worse, it can lead to the patient misunderstanding the doctor’s orders. Therefore, punctuation after ASR to enhance readability is an indispensable step. Most recent work has been fine-tuning downstream tasks on pre-trained models, but the models lack knowledge of relevant domains. Furthermore, most of the research is based on the English language, and there is less work in Chinese and even less in the medical field. Based on this, we thought of adding Chinese medical data to the model pre-training stage and adding the task of punctuation restoration in this work. From this, we proposed the Punctuation Restoration Pre-training Mask Language Model (PRMLM) task in the pre-training stage and used contrastive learning at this stage to enhance the model effect. Then, we propose a Punctuation Prior Knowledge Fine-tuning (PKF) method to play the role of contrast learning better when fine-tuning the downstream punctuation restoration task. In our medical field dataset, we performed a series of comparisons with existing algorithms to verify the proposed algorithm’s effectiveness.
ASM: Adaptive Sample Mining for In-The-Wild Facial Expression Recognition

Given the similarity between facial expression categories, the presence of compound facial expressions, and the subjectivity of annotators, facial expression recognition (FER) datasets often suffer from ambiguity and noisy labels. Ambiguous expressions are challenging to differentiate from expressions with noisy labels, which hurt the robustness of FER models. Furthermore, the difficulty of recognition varies across different expression categories, rendering a uniform approach unfair for all expressions. In this paper, we introduce a novel approach called Adaptive Sample Mining (ASM) to dynamically address ambiguity and noise within each expression category. First, the Adaptive Threshold Learning module generates two thresholds, namely the clean and noisy thresholds, for each category. These thresholds are based on the mean class probabilities at each training epoch. Next, the Sample Mining module partitions the dataset into three subsets: clean, ambiguity, and noise, by comparing the sample confidence with the clean and noisy thresholds. Finally, the Tri-Regularization module employs a mutual learning strategy for the ambiguity subset to enhance discrimination ability, and an unsupervised learning strategy for the noise subset to mitigate the impact of noisy labels. Extensive experiments prove that our method can effectively mine both ambiguity and noise, and outperform SOTA methods on both synthetic noisy and original datasets. The supplement material is available at https://github.com/zzzzzzyang/ASM .
IPFR: Identity-Preserving Face Reenactment with Enhanced Domain Adversarial Training and Multi-level Identity Priors

In the face reenactment task, identity preservation is challenging due to the leakage of driving identity and the complexity of source identity. In this paper, we propose an Identity-Preserving Face Reenactment (IPFR) framework with impressive expression and pose transfer. To address the leakage of driving identity, we develop an enhanced domain discriminator to eliminate the undesirable identity in the generated image. Considering the complexity of source identity, we inject multi-level source identity priors to keep the identity domain of generated image close to that of source. In detail, firstly, we utilize a 3D geometric prior from the 3D morphable face model (3DMM) to control face shape and reduce artifacts caused by occlusion in the module of generating motion field; secondly, we use an identity texture prior extracted by face recognition network to supervise the final stage, aiming to make the identity domain of the generated close to that of source. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art methods on image quality and identity preservation. Ablation studies are also conducted to further validate the effectiveness of our individual components.
Dynamic Multi-modal Prompting for Efficient Visual Grounding

Prompt tuning has emerged as a flexible approach for adapting pre-trained models by solely learning additional inputs while keeping the model parameters frozen. However, simplistic prompts are insufficient to effectively address the challenges posed by complex multi-modal tasks such as visual grounding. In this paper, we propose a novel prompting architecture called Dynamic Multi-modAl Prompting (DMAP) for visual grounding. DMAP incorporates input-dependent prompting to tailor instance-level prompts for more accurate representation and dynamic multi-modal prompting to capture the relationship between the textual and visual inputs. To this end, we design a Dynamic Prompt Network (DPN) to generate multi-modal prompts based on the specific inputs, enhancing both adaptive prompt generation and multi-modal feature fusion. Extensive experimental results demonstrate the superiority of DMAP over competing methods in parameter-efficient settings. Furthermore, DMAP consistently outperforms state-of-the-art VG methods even when fine-tuning all parameters.
MTQ-Caps: A Multi-task Capsule Network for Blind Image Quality Assessment

Blind image quality assessment (BIQA) is a field that aims to predict the quality of images without the use of reference images. This field has garnered considerable attention due to its potential applications in areas such as visual understanding and computational imaging. The two primary challenges in BIQA are the differentiation of distortion types and the prediction of quality scores. However, previous approaches typically address only one of these tasks, neglecting the associations between them. In this paper, we propose a multi-task capsule network for BIQA (MTQ-Caps). The proposed MTQ-Caps includes two types of quality capsules routed from CNN-based features: distortion capsules, which correspond to specific distortion types, and content capsules, which encapsulate semantic information for quality score estimation. Consequently, MTQ-Caps utilizes both synthetic distortions and human subjective judgments in a multi-task learning approach. Experimental results demonstrate that our proposed method outperforms state-of-the-art methods on synthetic databases and achieves competitive performance on authentic databases, even without distortion descriptions.
Pyramid Shape-Aware Semi-supervised Learning for Thyroid Nodules Segmentation in Ultrasound Images

The accurate segmentation of thyroid nodules in ultrasound (US) images is critical for computer-aided diagnosis of thyroid cancer. While the fully supervised methods achieve high accuracy, they require a significant amount of annotated data for training, which is both costly and time-consuming. Semi-supervised learning can address this challenge by using a limited amount of labeled data in combination with a large amount of unlabeled data. However, the existing semi-supervised segmentation approaches often fail to account for both geometric shape constraints and scale differences of objects. To address this issue, in this paper we propose a novel Pyramid Shape-aware Semi-supervised Learning (PSSSL) framework for thyroid nodules segmentation in US images, which employs a dual-task pyramid prediction network to jointly predict the Segmentation Maps (SEG) and Signed Distance Maps (SDM) of objects at different scales. Pyramid feature prediction enables better adaptation to differences in nodule size, while the SDM provides a representation that encodes richer shape features of the target. PSSSL learns from the labeled data by minimizing the discrepancy between the prediction and the ground-truth and learns from unlabeled data by minimizing the discrepancy between the predictions at different scales and the average prediction. To achieve reliable and robust segmentation, two uncertainty estimation modules are designed to emphasize reliable predictions while ignoring unreliable predictions from unlabeled data. The proposed PSSSL framework achieves superior performance in both quantitative and qualitative evaluations on the DDTI and TN3k datasets to State-Of-The-Art semi-supervised approaches. The code is available at https://github.com/wuliZN2020/Thyroid-Segmentation-PSSSL .
L2MNet: Enhancing Continual Semantic Segmentation with Mask Matching

Continual semantic segmentation (CSS) aims to continuously learn a semantic segmentation model that incorporates new categories while avoiding forgetting the previously seen categories. However, CSS faces a significant challenge known as weight shift, which leads to the network mistakenly predicting masks belonging to new categories instead of their actual categories. To mitigate this phenomenon, we propose a novel module named mask matching module, which transfers pixel-level prediction task into a mask-level feature matching task by computing the similarity between mask features and prototypes. Further, we introduce a new paradigm and a network called Learn-to-Match (L2M) Net, which alleviates weight shift and gains remarkable improvements on long settings by leveraging mask-level feature matching. Our method can be easily integrated into various network architectures without extra memory and data cost. Experiments conducted on the Pascal-VOC 2012 and ADE20K datasets demonstrate that, particularly on long settings where CSS encounters more challenging settings, our method achieves a remarkable $$10.6\%$$ improvement in terms of all mean Intersection over Union (mIoU) and establishes a new state-of-the-art performance in the demanding CSS settings.
Learning Efficient Representations for Image-Based Patent Retrieval

Patent retrieval has been attracting tremendous interest from researchers in intellectual property and information retrieval communities in the past decades. However, most existing approaches rely on textual and metadata information of the patent, and content-based image-based patent retrieval is rarely investigated. Based on traits of patent drawing images, we present a simple and lightweight model for this task. Without bells and whistles, this approach significantly outperforms other counterparts on a large-scale benchmark and noticeably improves the state-of-the-art by 33.5% with the mean average precision (mAP) score. Further experiments reveal that this model can be elaborately scaled up to achieve a surprisingly high mAP of 93.5%. Our method ranks first in the ECCV 2022 Patent Diagram Image Retrieval Challenge.
Physical laboratory for students practical laboratory works of The Imperial Moscow University from the moment of foundation until 1917

The present survey, timed to a century and a half anniversary of practical laboratory works in Physics being introduced in the curriculum of The Imperial University of Moscow for the students of the Faculty of Physics, provides a broad coverage of its functioning in the period from 1873 to 1917. The author gives a brief overview of the history of its founding, includes the data concerning the development of its equipment infrastructure, stuffing and figures about the number of students who took physics labs in those years. The contents of practical laboratory works are being analysed. The author vividly shows that professor A. P. Sokolov’s “Guide to exercises in general physics labs of Physical Institute of The Imperial University of Moscow” relies heavily on “Guide to Practical Classes in Physics” by F. Kohlrausch. The author familiarizes the readers with some historical references, which enable to better enhance the readers’ understanding of the details of practical training of physics students in The Imperial University of Moscow at the end of the 19th and the beginning of the 20th century. The article also contains rare historical pictures, some of which are published for the first time.
Mobile Assisted Language Learning Needs Assessment in Secondary Education

Mobile-assisted language learning (MALL) has entered students' educational life with a rising momentum. For this reason, it is necessary to determine the needs of English language curricula, teachers, and students in terms of mobile-assisted language learning. In this study, a convergent parallel design of mixed methods was used. The study group of research consists of 130 high school students and 5 teachers working in secondary education institutions. A mobile application adoption scale was used for the data collected from the students. Semi-structured interviews were conducted with teachers. As a result of the research, it was concluded that students find mobile applications useful and adopt them, and teachers need to be informed about the MALL. It was also found that mobile applications should support the target learning outcomes and cover all four language skills.
Cyber Laws and Media Censorship in Pakistan: An Investigation of Governmental Tactics to Curtail Freedom of Expression and Right to Privacy

All the democracies in the world have universally recognised the right to privacy and freedom of expression as fundamental human rights. It is fundamental to the preservation of all human rights, a basic element of democracy and a vital part of human dignity. Governments and governmental agencies frequently violate internationally accepted human rights, norms and standards under the cover of legislation. Media censorship is a worldwide issue that has existed for centuries. Censorship is often justified on the grounds of maintaining public order, but the underlying motive is to keep the people uninformed of governmental activities. It is argued that civil society, media personnel and the common people are working under pressure in Pakistan due to cyber and media laws. In the name of national security, the state has been working, intentionally, to put in place the best possible surveillance systems to establish a sort of watchdog over activists, bloggers, journalists and the general public.
AgileNet: A Rapid and Efficient Breast Lesion Segmentation Method for Medical Image Analysis

Current medical image segmentation approaches have shown promising results in the field of medical image analysis. However, their high computational demands pose significant challenges for resource-constrained medical applications. We propose AgileNet, an efficient breast lesion segmentation that achieves a balance between accuracy and efficiency by leveraging the strengths of both convolutional neural networks and transformers. The proposed Agile block facilitates efficient information exchange by aggregating representations in a cost-effective manner, incorporating both global and local contexts. Through extensive experiments, we demonstrate that AgileNet outperforms state-of-the-art models in terms of accuracy, model size, and throughput when deployed on resource-constrained devices. Our framework offers a promising solution for achieving accurate and efficient medical image segmentation in resource-constrained settings.
Where to Focus: Central Attention-Based Face Forgery Detection

Face forgery detection in compressed images is an active area of research. However, previous frequency-based methods are subject to two limitations. One aspect to consider is that they apply the same weight to different frequency bands. Moreover, they exhibit an equal treatment of regions that contain distinct semantic information. To address these limitations above, we propose the Central Attention Network (CAN), a multi-modal architecture comprising two bright components: Adaptive Frequency Embedding (AFE) and Central Attention (CA) block. The AFE module adaptively embeds practical frequency information to enhance forged traces and minimize the impact of redundant interference. Moreover, the CA block can achieve fine-grained trace observation by concentrating on facial regions where indications of forgery frequently manifest. CAN is efficient in extracting forgery traces and robust to noise. It effectively reduces the unnecessary focus of our model on irrelevant factors. Extensive experiments on multiple datasets validate the advantages of CAN over existing state-of-the-art methods.
Exploring a Distillation with Embedded Prompts for Object Detection in Adverse Environments

Efficient and robust object detection in adverse environments is crucial and challenging for autonomous agents. The current mainstream approach is to use image enhancement or restoration as a means of image preprocessing to reduce the domain shift between adverse and regular scenes. However, these image-level methods cannot guide the model to capture the spatial and semantic information of object instances, resulting in only marginal performance improvements. To overcome this limitation, we explore a Prompts Embedded Distillation framework, called PED. Specifically, a spatial location prompt module is proposed to guide the model to learn the easily missed target position information. Considering the correlation between object instances in the scene, a semantic mask prompt module is proposed to constrain the global attention between instances, making each aggregated instance feature more discriminative. Naturally, we propose a teacher model with embedded cues and finally transfer the knowledge to the original student model through focal distillation. Extensive experimental results demonstrate the effectiveness and flexibility of our approach.
DFR-ECAPA: Diffusion Feature Refinement for Speaker Verification Based on ECAPA-TDNN

Diffusion Probabilistic Models have gained significant recognition for their exceptional performance in generative image modeling. However, in the field of speech processing, a large number of diffusion-based studies focus on generative tasks such as speech synthesis and speech conversion, and few studies apply diffusion models to speaker verification. We investigated the integration of the diffusion model with the ECAPA-TDNN model. By constructing a dual-network branch architecture, the network further extracts and refines speaker embeddings under the guidance of the intermediate activations of the pre-trained DDPM. We put forward two methods for fusing network branch features, both of which demonstrated certain improvements. Furthermore, our proposed model also provides a new solution for semi-supervised cross-domain speaker verification. Experiments on Voxceleb and CN-Celeb show that DFR-ECAPA outperform origin ECAPA-TDNN by around 20%.
Few-Shot Object Detection via Classify-Free RPN

The research community has shown great interest in few-shot object detection, which focuses on detecting novel objects with only a small number of annotated examples. Most of the works are based on the Faster R-CNN framework. However, due to the absence of annotated data for novel instances, models are prone to base class bias, which can result in misclassifying novel instances as background or base instances. Our analysis reveals that although the RPN is class-agnostic in form, the binary classification loss possesses class-awareness capabilities, which can lead to the base class bias issue. Therefore, we propose a simple yet effective classify-free RPN. We replace the binary classification loss of the RPN with Smooth L1 loss and adjust the ratio of positive and negative samples for computing the loss. This avoids treating anchors matched with novel instances as negative samples in loss calculation, thereby mitigating the base class bias issue. Without any additional computational cost or parameters, our method achieves significant improvements compared to other methods on the PASCAL VOC and MS-COCO benchmarks, establishing state-of-the-art performance.
Hierarchical Spatial-Temporal Network for Skeleton-Based Temporal Action Segmentation

Skeleton-based Temporal Action Segmentation (TAS) plays an important role in analyzing long videos of motion-centered human actions. Recent approaches perform spatial and temporal information modeling simultaneously in the spatial-temporal topological graph, leading to high computational costs due to the large graph magnitude. Additionally, multi-modal skeleton data has sufficient semantic information, which has not been fully explored. This paper proposes a Hierarchical Spatial-Temporal Network (HSTN) for skeleton-based TAS. In HSTN, the Multi-Branch Transfer Fusion (MBTF) module utilizes a multi-branch graph convolution structure with an attention mechanism to capture spatial dependencies in multi-modal skeleton data. In addition, the Multi-Scale Temporal Convolution (MSTC) module aggregates spatial information and performs multi-scale temporal information modeling to capture long-range dependencies. Extensive experiments on two challenging datasets are performed and our proposed method outperforms the State-of-the-Art (SOTA) methods.
Mobile-LRPose: Low-Resolution Representation Learning for Human Pose Estimation in Mobile Devices

Human pose estimation has made great progress in performance due to the development of deep learning. Current methods, including some lightweight networks, usually generate high-resolution heatmaps with rich position information to ensure high accuracy, however, the computational cost is heavy and sometimes unacceptable to mobile devices. In this paper, we construct a network backbone based on the modified MobileNetV2 to only generate low-resolution representations. Then, to enhance the capability of keypoints localization for our model, we also make crucial improvements consisting of bottleneck atrous spatial pyramid, local-space attention, coordinate attention and position embedding. In addition, we design two different network heads for 2D and 3D pose estimation to explore the extensibility of the backbone. Our model achieves superior performance to state-of-the-art lightweight 2D pose estimation models on both COCO and MPII datasets, which achieves 25+ FPS on HUWEI Kirin 9000 and outperforms MoveNet in the same device. Our 3D model also makes nearly 50% and 90% reduction on parameters and FLOPs compared to lightweight alternatives. Code is available at: https://github.com/NanXinyu/Mobile_LRPose.git .