Reviewerly/exHarmony huggingface dataset:
Mask-Guided Transformer for Human-Object Interaction Detection

Human-object interaction (HOI) detection is a meaningful research topic on human activity understanding. Recent works have made significant progress by focusing on efficient triplet matching and leveraging image-wide features based on encoder-decoder architecture. However, the ability to gather relevant contextual information about human is limited and different sub-tasks in HOI detection are not differentiated by specific decoupling in previous methods. To this end, we propose a new transformer-based method for HOI detection, namely, Mask-Guided Transformer (MGT). Our model, which is composed of five parallel decoders with a shared encoder, not only emphasizes interactive regions by applying body features, but also disentangles the prediction of instance and interaction. We achieve a favorable result at 63.3 mAP on the well-known HOI detection dataset V-COCO.
Real Time Face Mask Detection Using Tiny YOLOv3

To stop the Covid-19 from spreading different techniques were developed and implemented. To make the use of face mask was effective to control the spread of it, as it transmits from one person to other. The use of face mask correctly is also important. The main objective is to develop a technique which can be used to monitor the people wearing face mask, not wearing the face mask or incorrect face mask and also to make use of the model which can be used on portable devices without the use of GPU for real time detection. The Yolov3-tiny model was used for detection. It achieved the mean average precision of 68% and it runs at 13 FPS without the use of GPU. The model was tested for image detection and also for real time detection.
Visual Analysis Motivated Super-Resolution Model for Image Reconstruction

This paper presents a concise end-to-end visual analysis motivated super-resolution model VASR for image reconstruction. Compatible with the existing machine vision feature coding framework, the features extracted from the machine vision task model are super-resolution amplified to reconstruct the original image for human vision. The experimental results show that without additional bit-streams, VASR can well complete the task of image reconstruction based on the extracted machine features, and has achieved good results on COCO, OpenImages, TVD, and DIV2K datasets.
On Pre-chewing Compression Degradation for Learned Video Compression

Artificial Intelligence (AI) needs huge amounts of data, and so does Learned Restoration for Video Compression. There are two main problems regarding training data. 1) Preparing training compression degradation using a video codec (e.g., Versatile Video Coding - VVC) costs a considerable resource. Significantly, the more Quantization Parameters (QPs) we compress with, the more coding time and storage are required. 2) The common way of training a newly initialized Restoration Network on pure compression degradation at the beginning is not effective. To solve these problems, we propose a Degradation Network to pre-chew (generalize and learn to synthesize) the real compression degradation, then present a hybrid training scheme that allows a Restoration Network to be trained on unlimited videos without compression. Concretely, we propose a QP-wise Degradation Network to learn how to compress video frames like VVC in real-time and can transform the degradation output between QPs linearly. The real compression degradation is thus pre-chewed as our Degradation Network can synthesize the more generalized degradation for a newly initialized Restoration Network to learn easier. To diversify training video content without compression and avoid overfitting, we design a Training Framework for Semi-Compression Degradation (TF-SCD) to train our model on many fake compressed videos together with real compressed videos. As a result, the Restoration Network can quickly jump to the near-best optimum at the beginning of training, proving our promising scheme of using pre-chewed data for the very first steps of training. In other words, a newly initialized Learned Video Compression can be warmed up efficiently but effectively with our pre-trained Degradation Network. Besides, our proposed TF-SCD can further enhance the restoration performance in a specific range of QPs and provide a better generalization about QPs compared with the common way of training a restoration model. Our work is available at https://minhmanho.github.io/prechewing_degradation.
Intelligent Human-machine Game System Based on Machine Vision

This paper designs an intelligent man-machine game system based on machine vision, including image acquisition and processing, game algorithm, and intelligent control of robotic arm. The camera collects the chessboard and chess piece images, performs image processing through OpenCV, uses the Hough transform algorithm to find the position of the board and pieces, and extracts the chess game information contained in the picture. The chess game information is entered into the game algorithm, and after processing, the next position of the intelligent system is obtained. Finally, the drop position is sent to the Arduino, where the Arduino controls the robotic arm drop. The system realizes the intelligent control of the robotic arm based on machine vision and completes the function of man-machine game.
Semantic Attribute Guided Image Aesthetics Assessment

Image aesthetics assessment (IAA) measures the perceived beauty of images using a computational approach. People usually assess the aesthetics of an image according to semantic attributes, e.g., lighting, color, object emphasis, etc. However, the state-of-the-art IAA approaches usually follow the data-driven framework without considering the rich attributes contained in images. With this motivation, this paper presents a new semantic attribute guided IAA model, where the attention maps of semantic attributes are employed to enhance the representation ability of general aesthetic features for more effective aesthetics assessment. Specifically, we first design an attribute attention generation network to obtain the attention maps for different semantic attributes, which are utilized to weight the general aesthetic features, producing the semantic attribute-enhanced feature representations. Then, the Graph Convolutional Network (GCN) is employed to further investigate the inherent relationship among the enhanced aesthetic features, producing the final image aesthetics prediction. Extensive experiments and comparisons on three public IAA databases demonstrate the effectiveness of the proposed method.
No Reference Stereoscopic Video Quality Assessment based on Human Vision System

In this paper, we propose a no-reference stereoscopic video quality assessment (NR-SVQA) based on human vision system (HVS). Firstly, we build a frequency transform module (FTM), which maps spatial domain to frequency domain by cosine discrete transform (DCT), and selects important frequency components through channel attention mechanism. Secondly, we use dynamic convolution to regionally process the same input. Thirdly, we use convolutional long short term memory (Conv-LSTM) to extract spatio-temporal information rather than just temporal information. Finally, in order to better simulate the visual characteristics of human eyes, we build a optic chiasm module. The experiment results show that our method outperforms any other methods.
Low Light RAW Image Enhancement Using Paired Fast Fourier Convolution and Transformer

Compared to RGB images non-linearly mapped from RAW data through the Image Signal Processor (ISP), RAW data are linear to scene radiance and contain more native information, which is better to be modeled in many vision tasks. This work proposes to enhance low-light images in the RAW domain via a cross-scale framework using paired Fast Fourier Convolution (FFC) and Transformer, driving the network to characterize images effectively. The entire framework has three scales to abstract low-level, mid-level, and high-level representations of input images. We embed paired FFC and Transformer in each scale to attain spatial-spectral information extraction and aggregation. Specifically, by transforming features from the spatial domain into the spectral domain with FFC, pixel correlations can be effectively exploited locally and globally, generating representative features for the input image. Immediately, the Transformer using multi-head self-attention mechanism is applied to aggregate and embed important features. Experimental results demonstrate that our method significantly outperforms state-of-the-art low-light enhancement works in both full reference assessment metrics, including PSNR, MPSNR, and SSIM, and no-reference metrics, such as NIMA. Meanwhile, the perceptual quality of the proposed method is more visually pleasing than that of other methods.
CFNet: A Coarse-to-Fine Network for Few Shot Semantic Segmentation

Since a huge amount of datasets is required for semantic segmentation, few shot semantic segmentation has attracted more and more attention of researchers. It aims to achieve semantic segmentation for unknown categories from only a small number of annotated training samples. Existing models for few shot semantic segmentation directly generate segmentation results and concentrate on learning the relationship between pixels, thus ignoring the spatial structure of features and decreasing the model learning ability. In this paper, we propose a coarse-to-fine network for few shot semantic segmentation, named CFNet. Firstly, we design a region selection module based on prototype learning to select the approximate region corresponding to the unknown category of the query image. Secondly, we elaborately combine the attention mechanism with the convolution module to learn the spatial structure of features and optimize the selected region. For the attention mechanism, we combine channel attention with self-attention to enhance the model ability of exploring the spatial structure of features and the pixel-wise relationship between support and query images. Experimental results show that CFNet achieves 65.2% and 70.1% in mean-IoU (mIoU) on PASCAL-5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> for 1-shot and 5-shot settings, respectively, and outperforms state-of-the-art methods by 1.0%.
A Large-scale Sports Tracking Dataset and Progressive Re-detection Based Sports Tracking

Recent years have witnessed the great progress of Visual Object Tracking (VOT) which aims to predict the position of an object in each video frame given only its initial appearance. However, even the state-of-the-art methods are confronted with performance degradation, i.e., the tracker drift problem, in sports video scenes (e.g., soccer, basketball). There are two main causes that should be responsible for the tracker drift problem. First, the object of interest is often occluded by other objects that share a similar appearance. Such severe occlusion prevents the model from distinguishing the correct tracking object from other distractors in the future frames. Second, in sports videos, the objects often move fast from one place to another, which incurs severe blurry visual effects among consecutive frames. To address the issues of the tracker drift problem, we treat VOT as a tracking-by-re-detection task. Specifically, we detect candidate objects within a searching area (determined by object location in the previous frame) in the current frame and develop a progressive algorithm to filter out distractors in the area, which proves robust towards occlusion scenarios and tracker drift problems. Combining the advantages of our settings, the proposed framework method is robust to motion blur and object occlusion issues and achieves state-of-the-art tracking results on our challenging dataset.
Weakly Supervised Region-Level Contrastive Learning for Efficient Object Detection

Semi-supervised learning, which assigns pseudo labels with models trained using limited labeled data, has been widely used in object detection to reduce the labeling cost. However, the provided pseudo annotations inevitably suffer noise since the initial model is not perfect. To address this issue, this paper introduces contrastive learning into semi-supervised object detection, and we claim that contrastive loss, which inherently relies on data augmentations, is much more robust than traditional softmax regression for noisy labels. To take full advantage of it in the detection task, we incorporate labels prior to contrastive loss and leverage plenty of region proposals to enhance diversity, which is crucial for contrastive learning. In this way, the model is optimized to make the region-level features with the same class be translation and scale invariant. Furthermore, we redesign the negative memory bank in contrastive learning to make the training more efficient. As far as we know, we are the first attempt that introduces contrastive learning in semi-supervised object detection. Experimental results on detection benchmarks demonstrate the superiority of our method. Notably, our method achieves 79.9% accuracy on VOC, which is 6.2% better than the supervised baseline and 0.7% improvement compared with the state-of-the-art method.
Vehicle Detection of Autonomous Sanitation Robot in Campus

Under the COVID-19 and other terrible environments workers are constrained to sweep campus and public area. Intelligent and driverless sanitation robot can solve the problem. Obstacle avoidance and garbage cleanup are its important functions. Based on the driverless sanitation robot project introduced by Sanda University, this paper carries out recognition of campus vehicles and improves its obstacle avoidance function. Through image processing, the object features of different environment and climate conditions are extracted, analyzed and recognized, so as to achieve more accurate recognition of campus vehicles. And opencv and python language are used to complete the implementation of vehicle detection.
Development of Intelligent Transportation Function Based on NAO Robot

This study is based on NAO robot to develop a series of transportation functions to assist people's daily life. Through studying the python library corresponding to Choregraph platform and NAO robot, the design of robot's hand movements and foot movements, the research of identifying objects by machine vision, ranging algorithm and color recognition, testing the success of each function development, and finally integrating a series of functions. I hope I can make a certain contribution to the research of NAO robot.
Design and Implementation of “Angel” Empty Nest Elderly Guardian System

In view of the fact that the empty nest elderly can not be treated in time, a health management guardian system based on smart devices is designed. The system relies on the smart bracelet and App to carry out real-time monitoring of the elderly's health, so as to achieve the purpose of controlling the elderly's physical status and timely emergency treatment. The system is divided into three parts: acquisition smart bracelet, cloud server and guardian App. The guardian App includes the children's end and the elderly end, both of which are carried out at the same time to protect the healthy life of the elderly. The system can be connected with the community medical center, so the elderly can call for help at the first time in case of an accident, whether at home or outside. The method in the paper solves the health problems of empty nest elderly, and has certain novelty and practical value.
Implementation of Mask Wearing Recognition

Because of the pandemic of COVID-19 since 2020, it seriously affects people's daily life and causes huge economic loss. Recently, the international community has mostly adopted an attitude of coexisting with Covid-19. We cannot ignore the harm the virus can bring to us. In order to effectively protect everyone from the virus, the most basic and effective way is to wear a mask to keep you away from exposure to the virus when going to public areas. Vision intelligence can play an important role in public health issues. In this paper, we utilize the object detection method to implement an actual mask wearing recognition system which can detect if people have a face mask on their face, and send a warning message if not wearing a mask. YOLOv3 is the basic framework for our implementation. After training and fine-tuning processes, the implemented model can perform effectively and correctly.
Research on Service Model of Mobile Library Based on Context Awareness

The mobile library perceives the user's situation and provides personalized information services as the goal. The paper starts with situational awareness as an entry point, starting from two perspectives of mobile phone user situational awareness middleware and service model level, researches and proposes the service model framework of library situational awareness, design context-aware adaptive middleware, construct a mobile library mobile service model based on context-awareness, design a personalized learning system incorporating context-awareness which can improve and enrich the theory of mobile library knowledge service research. It provides a new basis for the development and application of the theory of library situational awareness services.
Real Time Comparative Analysis between Solar Fixed Plate and Tracking Methodology

The “straight beam” carries around 90% of the solar energy, while the “diffuse sunlight” carries the remaining 10%. Since the sun's energy is most concentrated in its direct beam, keeping the panels in the sun's line of sight for as long as possible increases their efficiency. The energy generated by a direct beam of light is diminished by the cosine angle between both the source of light & the panel. Reflectance is essentially constant for angles of incidence up to about 50°, and then it drops off sharply. While a stationary flat panel can be aligned to capture the majority of the sun's rays at midday, significant power is also accessible in the early morning and late afternoon, when this misalignment makes it impractical to do so. As an illustration, the available energy is already reduced by half even when the Sun is only 10 degrees above the horizon, compared to when it is directly above. The fundamental advantage of a tracking system is to collect energy from the sun for the greatest period of the day and with the most exact alignment since the Sun's position changes throughout the year. An appropriate motor will then be selected after a comparison of the model's efficiency and the output from solar panel model.
Identification of Non-Linear Dynamic systems applying Unscented H-Infinity Adaptive Filtering approach

This study proposes an adaptive filtering approach for nonlinear system identification employing a functional link artificial neural network (FLANN) including the extended H infinity filter (EHF) and the unscented H infinity filter (UHF). By identifying the system and state model based on the FLANN model's structure and estimating its parameters, the nonlinear system is identified. The desired output of the system is compared with original and yields the mean square error (MSE) using EHF and UHF. The performance is compared with extended Kalman filter (EKF), unscented Kalman filter (UKF) and many state of art methods and it is found that UHF is a better performer in terms of MSE. By incorporating the unscented transform principle into the EHF technique, the linear statistical error propagation strategy may be involved to perform the unscented H-infinity filtering. The suggested strategy is resistant against model uncertainty and can produce better accurate identification results with less sophisticated processing. Its efficiency is demonstrated by three simulated cases.
Space and Level Cooperation Framework for Pathological Cancer Grading

Clinically, the pathological images are intuitive for cancer diagnosis and have been considered as the ‘gold standard’. There are two challenges for applying deep learning into the pathological images analysis: the ultra-large size and the noisy annotations. A pathological image usually contains billions of pixels, which is unsuitable for normal classification models. Furthermore, the ultra-large size and mixed cancerous cells compel the doctor to draw rough boundaries of lesion area according to the cancerous level, which brings two kinds of noisy labels: space noise (annotating inaccurate scope of cancerous area) and level noise (annotating inaccurate cancerous level). Based on the above findings, we propose the space and level cooperation framework, comprising a space-aware branch and a level-aware branch, for pathological cancer grading with noisy annotations. The space-aware branch first turns the ultra-large image into a Multilayer Superpixel (MS) graph, significantly reducing the size and preserving the global features. Then, a global-to-local rectifying strategy is adopted to solve the space noise. The level-aware branch adopts different grouped kernels and a novel grading loss function to handle level noise. Mean-while, two branches cooperate through complementing missing features of each other for handling the above two challenges. Extensive experiments demonstrate that with noisy annotations, the proposed framework achieves SOTA performance on our HCC dataset and two public datasets.
Boids Flocking Algorithm for Situation Assessment of Driver Assistance Systems

Driver assistance systems are increasingly becoming part of the standard equipment of vehicles and thus contribute to road safety. However, as they become more widespread, the requirements for cost efficiency are also increasing, and so few and inexpensive sensors are used in these systems. Especially in challenging situations, this leads to the fact that target discrimination cannot be ensured which may lead to false reactions of the driver assistance system. In this paper, the Boids flocking algorithm is used to generate semantic neighborhood information between tracked objects which in turn can significantly improve the overall performance. Two different variants were developed: First, a freemoving flock whereby a separate flock is generated per tracked object and second, a formation-controlled flock where boids of a single flock move along the future road course in a pre-defined formation. In the first approach, the interaction between the flocks as well as the interaction between the boids within a flock is used to generate additional information, which in turn can be used to improve, for example, lane change detection. For the latter approach, new behavioral rules have been developed, so that the boids can reliably identify control-relevant objects to a driver assistance system. Finally, the performance of the presented methods is verified through extensive simulations.
Visually Encrypted Watermarking for Ordered-Dithered Clustered-Dot Halftones

Visual encryption and show through watermarking are widely used techniques to hide secret data in halftone images. The secret watermark can be quickly revealed when the halftone images are printed in transparency and overlaid on each other. The present studies emphasize developing a show-through watermarking technique for the clustered-dot halftone types. The proposed method exploits the properties of various configurations of dither array screens constructed using distinct Gaussian filters to embed watermarks. Two approaches are proposed, i.e., adjacent dither array pairs and the dither array translations. The optimal configuration of dither array parameters is developed to obtain maximum contrast and imperceptibility. Moreover, a new edge screen dithering is proposed to eliminate the edge artifacts, resulting in smooth screen transitions. As efficient ordered dithering is adopted for the watermark embedding, the joint watermarking and halftoning can be performed without additional computations. In comparison to the existing state-of-the-art show-through watermarking techniques, the proposed method can present superior image quality, computational simplicity, and decoded watermarks can be perceived with clear contrast when the images are overlaid with each other.