Reviewerly/exHarmony huggingface dataset:
Lightweight Real-Time Polyp Segmentation Network Based on Factorized Convolution Unit

The current polyp segmentation network generally uses ResNet and Res2Net as the backbone extraction network, which improves segmentation accuracy. However, due to its high computational burden and hundreds or thousands of feature channels, it limits the deployment of the network on devices with limited computing power, such as embedded devices and mobile devices. Based on the above requirements, a lightweight real-time polyp segmentation network is proposed, which uses factorized convolution unit as the basic extraction unit of the backbone network, which dramatically reduces the computational complexity and the number of parameters while maintaining the segmentation accuracy. Specifically, the proposed model has only 679,000 parameters and runs at 73FPS in a single RTX 3060 GPU to achieve real-time segmentation. Experiments show that the method in this paper achieves the trade-off of speed, network size, and accuracy in Kvasir-SEG and ClinicDB datasets, with IoU reaching 81.72% and 80.41%, respectively
Overview of the development of AI dataset annotation

With the continuous development of artificial intelligence, various deep learning algorithms need a lot of training of annotated data, and how to improve the efficiency of data annotation has become a research hotspot. This paper analyzes the development history of AI data set annotation, summarizes the general framework of AI data set annotation, summarizes three semi-automatic or automatic AI data set annotation methods, and compares and analyzes the advantages and disadvantages of the three methods.
Motion Deblur with Non-Local Attention Network

Motion blur, which degrades image quality significantly, is a common and huge obstacle in many other image processing applications. And deep learning has been used in several fields of image processing in recent years. In this paper, we present an efficient motion deblur network based on the Non-Local Attention Network. This network can deblur an image blurred by motion blindly without any prior knowledge. Our network follows the encoder-decoder structure, and a residual network module consisting of multiple residual networks is added to both the encoder and the decoder to extract the depth features of the input feature maps. Local and non-local attention modules built according to the residual network idea are also added to the network, which in turn improves the network's ability to capture long-term dependencies and allows us to build deeper networks to improve the expressiveness of the network. Experiments have shown that our method achieves quantitatively and visually comparable or better results than current leading methods.
Path Planning of Follow-up Transport Robot Based on Machine Vision

With the upgrading and innovation of the logistics industry, the requirements for the level of transportation smart technologies continue to increase. The outbreak of the COVID-19 has further promoted the development of unmanned transportation machines. Aimed at the requirements of intelligent following and automatic obstacle avoidance of mobile robots in dynamic and complex environments, this paper uses machine vision to realize the visual perception function, and studies the real-time path planning of robots in complicated environment. And this paper proposes the Dijkstra-ant colony optimization (ACO) fusion algorithm, the environment model is established by the link viewable method, the Dijkstra algorithm plans the initial path. The introduction of immune operators improves the ant colony algorithm to optimize the initial path. Finally, the simulation experiment proves that the fusion algorithm has good reliability in a dynamic environment.
A study of aspect-level sentiment analysis based on deep learning

The study of aspect-level sentiment analysis using deep learning methods is one of the more important research directions in the field of natural language processing in recent years. In this paper, we address the problem of insufficient extraction of deep semantic features in existing aspect-level sentiment analysis research, design and build a sentiment analysis model based on the pre-trained language model BERT, fuse BiLSTM and GCN deep learning methods, analyze the sentiment tendency on the collected product review dataset, design relevant experiments to compare in the same application scenario, and verify the effectiveness of the proposed model.
Helmet Wearing Detection System for Non-Motor Vehicle Riders Based on k210 and YOLOv3

Wearing helmets can effectively reduce the secondary injuries caused by traffic accidents, and it is necessary to detect the helmet wearing conditions of non-motor vehicle riders. This paper proposes a portable and low-cost rider helmet wearing detection system. The system adopts the lightweight network framework of YOLOv3, and deploys the algorithm on the K210 microcontroller. The innovation of this paper is that the calculation amount and model size of the traditional YOLOv3 algorithm model are much larger than the maximum scale supported by the K210 chip, and it is difficult to deploy on small embedded devices. We need to improve the traditional YOLOv3 algorithm. This paper adopts the method of replacing the backbone network to reduce the complexity of the model, so that the algorithm can be deployed on the K210 microcontroller. This paper conducts comparative training under two different backbone networks, and deploys the appropriate model on the k210 single-chip microcomputer, which reduces the detection cost.
Self-Supervised Learning from Untrimmed Videos via Hierarchical Consistency

Natural untrimmed videos provide rich visual content for self-supervised learning. Yet most previous efforts to learn spatio-temporal representations rely on manually trimmed videos, such as Kinetics dataset (Carreira and Zisserman 2017), resulting in limited diversity in visual patterns and limited performance gains. In this work, we aim to improve video representations by leveraging the rich information in natural untrimmed videos. For this purpose, we propose learning a hierarchy of temporal consistencies in videos, i.e., visual consistency and topical consistency, corresponding respectively to clip pairs that tend to be visually similar when separated by a short time span, and clip pairs that share similar topics when separated by a long time span. Specifically, we present a Hierarchical Consistency (HiCo++) learning framework, in which the visually consistent pairs are encouraged to share the same feature representations by contrastive learning, while topically consistent pairs are coupled through a topical classifier that distinguishes whether they are topic-related, i.e., from the same untrimmed video. Additionally, we impose a gradual sampling algorithm for the proposed hierarchical consistency learning, and demonstrate its theoretical superiority. Empirically, we show that HiCo++ can not only generate stronger representations on untrimmed videos, but also improve the representation quality when applied to trimmed videos. This contrasts with standard contrastive learning, which fails to learn powerful representations from untrimmed videos. Source code will be made available here.
PACMAN: Attacking ARM Pointer Authentication With Speculative Execution

Memory corruption vulnerabilities have resulted in numerous exploits and significant damage to computer systems. ARM Pointer Authentication is a memory corruption mitigation that attempts to mitigate these issues by cryptographically signing pointers at runtime. We present PACMAN, a novel attack methodology that can forge correct pointer signatures and bypass the protection of pointer authentication without causing any crashes using microarchitectural side channels. PACMAN removes the primary barrier to conducting control-flow hijacking attacks on a platform protected with pointer authentication. We built two proof-of-concept attacks showing that PACMAN works across privilege levels on the Apple M1 CPU. We have also released a suite of open-source tools to enable the community to perform future research on Apple Silicon devices.
A Joint Tagging Event Extraction with Attention Mechanism

Event extraction can extract event information from text, which is a very important information extraction task. Currently, most methods assume that there is at most one event in a sentence. However, in practice there may be one or more events in a sentence. Therefore, there may be information overlap between multiple events. To solve this problem, this paper proposes a novel joint learning framework, JointEE. First, the similarity function is used to measure the types of events present in the sentence, and then the sequence annotation model with the attention mechanism is used to jointly extract trigger and arguments. The paper is evaluated on a public event extraction dataset, FewFC. Experiments show that, compared with previous methods, JointEE achieves good results on the overlapping event extraction problem.
The Impact of COVID-19 and the October 17th Revolution on Employees’ Performances: Case of Lebanese Banks

This research aims to identify clients’ perception toward employees’ performances, attitude towards banks’ procedures and to review their consideration of switching banks due to the October 17th revolution, and COVID-19 catastrophic outbreak. In this study, Lebanon was chosen as an ideal background and more specifically Lebanese Banks’ clients. This research opens the opportunity to test their level of approval of employees’ performance, banks’ procedures, and bank switching possibility. Data was collected, by gathering primary and secondary information, through a quantitative approach. Required data were gathered from surveying Lebanese clients dealing with different banks, and coming from diverse backgrounds and regions through an online survey. The main findings revealed that the October Revolution coupled with the outbreak of the Corona Virus had a tragic impact on Lebanese banks. Further results indicated that clients are not satisfied with employees’ performance. Consequently, the majority was considering switching banks in search of more satisfactory job performance by employees.
Multi-scale Contrastive Learning for Image Colorization

This paper proposes a multi-scale contrastive learning technique for image colorization. The image colorization task aims to find a map between the source gray image and the predicted color image. Multi-scale contrastive learning for unpaired image colorization has been proposed to transform the gray patches in the input image into the color patches in the output image. The contrastive learning method uses input and output patches and maximizes their mutual information to infer an efficient mapping between the two domains. We propose a multi-scale approach for contrastive learning where the contrastive loss is determined from different resolutions of the source image to improve the color quality. We further illustrate the effectiveness of the proposed approach through experimental outcomes and comparisons with cutting-edge strategies in image colorization tasks.
Low-Cost Hardware-Accelerated Vision-Based Depth Perception for Real-Time Applications

Depth estimation and 3D object detection are critical for autonomous systems to gain context of their surroundings. In recent times, compute capacity has improved tremendously, enabling computer vision and AI on the edge. In this paper, we harness the power of CUDA and OpenMP to accelerate ELAS (a stereoscopic vision-based disparity calculation algorithm) and 3D projection of the estimated depth while performing object detection and tracking. We also examine the utility of Bayesian inference in achieving real-time object tracking. Finally, we build a drive-by-wire car equipped with a stereo camera setup to test our system in the real world. The entire system has been made public and easily accessible through a Python module.
Context Unaware Knowledge Distillation for Image Retrieval

Existing data-dependent hashing methods use large backbone networks with millions of parameters and are computationally complex. Existing knowledge distillation methods use logits and other features of the deep (teacher) model and as knowledge for the compact (student) model, which requires the teacher’s network to be fine-tuned on the context in parallel with the student model on the context. Training teacher on the target context requires more time and computational resources. In this paper, we propose context unaware knowledge distillation that uses the knowledge of the teacher model without fine-tuning it on the target context. We also propose a new efficient student model architecture for knowledge distillation. The proposed approach follows a two-step process. The first step involves pre-training the student model with the help of context unaware knowledge distillation from the teacher model. The second step involves fine-tuning the student model on the context of image retrieval. In order to show the efficacy of the proposed approach, we compare the retrieval results, no. of parameters, and no. of operations of the student models with the teacher models under different retrieval frameworks, including deep cauchy hashing (DCH) and central similarity quantization (CSQ). The experimental results confirm that the proposed approach provides a promising trade-off between the retrieval results and efficiency. The code used in this paper is released publicly at https://github.com/satoru2001/CUKDFIR .
Moment Centralization-Based Gradient Descent Optimizers for Convolutional Neural Networks

Convolutional neural networks (CNNs) have shown very appealing performance for many computer vision applications. The training of CNNs is generally performed using stochastic gradient descent (SGD)-based optimization techniques. The adaptive momentum-based SGD optimizers are the recent trends. However, the existing optimizers are not able to maintain a zero mean in the first order moment and struggle with optimization. In this paper, we propose a moment centralization-based SGD optimizer for CNNs. Specifically, we impose the zero mean constraints on the first order moment explicitly. The proposed moment centralization is generic in nature and can be integrated with any of the existing adaptive momentum-based optimizers. The proposed idea is tested with three state-of-the-art optimization techniques, including Adam, Radam, and Adabelief on benchmark CIFAR10, CIFAR100, and TinyImageNet datasets for image classification. The performance of the existing optimizers is generally improved when integrated with the proposed moment centralization. Further, the results of the proposed moment centralization are also better than the existing gradient centralization. The analytical analysis using the toy example shows that the proposed method leads to a shorter and smoother optimization trajectory. The source code is made publicly available at https://github.com/sumanthsadhu/MC-optimizer .
TexIm: A Novel Text-to-Image Encoding Technique Using BERT

Often when we read some text, it leaves an impression in our mind. This perception imbibes the knowledge conveyed, the context, and the lexical information. Although there has been abundant research on the representation of text, research on devising techniques for visualization of embedded text is absent. Thus, we propose a novel “text-to-image” (TexIm) encoding enabling visualization of textual features. The proposed TexIm extracts the contextualized semantic and syntactic information present in the text through BERT and generates informed pictorial representations through a series of transformations. This unique representation is potent enough to assimilate the information conveyed, and the linguistic intricacies present in the text. Additionally, TexIm generates concise input representation that reduces the memory footprint by 37%. The proposed methodology has been evaluated on a hand-crafted dataset of Cricketer Biographies for the task of pair-wise comparison of texts. The conformity between the similarity of texts and the corresponding generated representations ascertain its fruitfulness.
Synthesis of Human-Inspired Intelligent Fonts Using Conditional-DCGAN

Despite numerous fonts already being designed and easily available online, the desire for new fonts seems to be endless. Previous methods focused on extracting style, shape, and stroke information from a large set of fonts, or transforming and interpolating existing fonts to create new fonts. The drawback of these methods is that generated fonts look-alike fonts of training data. As fonts are created from human handwriting documents, they have uncertainty and randomness incorporated into them, giving them a more authentic feel than standard fonts. Handwriting, like a fingerprint, is unique to each individual. In this paper, we have proposed GAN-based models that automate the entire font generation process, removing the labor involved in manually creating a new font. We extracted data from single-author handwritten documents and developed and trained class-conditioned DCGAN models to generate fonts that mimic the author’s handwriting style.
Automated Molecular Subtyping of Breast Cancer Through Immunohistochemistry Image Analysis

Molecular subtyping has a significant role in cancer prognosis and targeted therapy. However, the prevalent manual procedure for this has disadvantages, such as deficit of medical experts, inter-observer variability, and high time consumption. This paper suggests a novel approach to automate molecular subtyping of breast cancer using an end-to-end deep learning model. Immunohistochemistry (IHC) images of the tumor tissues are analyzed using a three-stage system to determine the subtype. A modified Res-UNet CNN architecture is used in the first stage to segregate the biomarker responses. This is followed by using a CNN classifier to determine the status of the four biomarkers. Finally, the biomarker statuses are combined to determine the specific subtype of breast cancer. For each IHC biomarker, the performance of segmentation models is analyzed qualitatively and quantitatively. In addition, the patient-level biomarker prediction results are also assessed. The findings of the suggested technique demonstrate the potential of computer-aided techniques to diagnose the subtypes of breast cancer. The proposed automated molecular subtyping approach can accelerate pathology procedures, considerably reduce pathologists’ workload, and minimize the overall cost and time required for diagnosis and treatment planning.
Balanced Sampling-Based Active Learning for Object Detection

Active learning focuses on building competitive models using less data by utilizing intelligent sampling, thereby reducing the effort and cost associated with manual data annotation. In this paper an active learning method based on a balanced sampling of images with high and low confidence object detection scores for training an object detector is proposed. Images with higher object prediction scores are sampled using the uncertainty measure proposed by Yu et al. which utilizes detection, classification and distribution statistics. Though this method encourages balanced distribution in sampling, a deeper look into the sampled distribution reveals that the under-represented classes in the initial labeled pool remain skewed throughout the subsequent active learning cycles. To mitigate this problem, in each active learning cycle, we propose to sample an equal proportion of images with high and low confidence object prediction scores from the model trained in the last cycle, where the low confidence prediction sample selection is based on the model’s prediction scores. Experiments conducted on the UEC Food 100 dataset show that the proposed method performs better than the baseline random sampling, CALD and low confidence prediction sampling method by +4.7, +8.7, and +3.1 mean average precision (mAP), respectively. Moreover, consistently superior performance of the proposed method is also demonstrated on the PASCAL VOC’07 and PASCAL VOC’12 datasets.
Yoga Pose Rectification Using Mediapipe and Catboost Classifier

Yoga will be unfruitful if the individual performing it does not have proper posture. Attending yoga classes and getting training sessions can be expensive and time-consuming, and most existing yoga-based applications are just focused on pose classification and posture correction for a small number of yoga poses, and most of them do not provide output in the form of audio. The dataset containing 13 (398 images) postures is analyzed with OpenCV, and the key points of the pose are retrieved using MediaPipe. These retrieved key points are used for calculating angles from the pose, and together the key points and angles are used to train machine learning models for distinguishing yoga postures. The improperly positioned body part was identified using the average values of each key point and computed angles. We were able to achieve 98.9% accuracy using CatBoost classifier. We have executed this model on 82 practical images and obtained 93.9% accuracy on yoga classification and 78.05% accuracy for identifying the improperly positioned body part.
Semi-supervised Semantic Segmentation for Effusion Cytology Images

Cytopathologists analyse images captured at different magnifications to detect the malignancies in effusions. They identify the malignant cell clusters from the lower magnification, and the identified area is zoomed in to study cell level details in high magnification. The automatic segmentation of low magnification images saves scanning time and storage requirements. This work predicts the malignancy in the effusion cytology images at low magnification levels such as $$10{\times }$$ and 4 $$\times $$ . However, the biggest challenge is the difficulty in annotating the low magnification images, especially the 4 $$\times $$ data. We extend a semi-supervised learning (SSL) semantic model to train unlabelled 4 $$\times $$ data with the labelled 10 $$\times $$ data. The benign F-score on the predictions of 4 $$\times $$ data using the SSL model is improved 15% compared with the predictions of 4 $$\times $$ data on the semantic 10 $$\times $$ model.
Local DCT-Based Deep Learning Architecture for Image Forgery Detection

A convolutional neural network model efficient in forgery detection in images regardless of the type of forgery is proposed. The AC coefficients in the block DCT of the entire image are analysed for the suspected forgery operation. The feature vector is extracted from the non-overlapping blocks of size $$8$$ $$\,\times \,$$ $$8$$ of the image. It consists of the standard deviation and non-zero counts of the block DCT coefficients of the image and its cropped version. The image is first converted to YCbCr colour space. The feature vector is extracted for all three channels. We then supply this feature vector as input to the deep neural network for detection. We have trained the DNN using CASIAv1 and CASIAv2 datasets separately and tested them. The train test ratio used is 80:20 for experimentation. Experimentation results on standard datasets, namely CASIA v1 and CASIA v2, reveal the efficiency of the proposed approach. A comparison with some of the existing approaches shows the proposed approach’s performance in terms of detection accuracy.