Reviewerly/exHarmony huggingface dataset:
End-to-End Point Cloud Geometry Compression and Analysis with Sparse Tensor

With the rapid development of deep learning, encoded objects such as images, videos, and point cloud objects are increasingly used in downstream tasks optimized by deep learning. Traditional coding tools are optimized for human perception, not machine vision. Therefore, we bring forward a point cloud lossy compression method for machine vision, which uses elaborate extracted features to ensure the point cloud classification accuracy. We present a multi-scale channel attention module, which can well integrate features of various channels and dimensions, ensuring the compression performance and integrating the upper-level semantic information well. The experimental results demonstrates that our method achieves 30% BD-Rate gains and 5% improvement in classification compared with PCGCV2 in Modelnet40.
Human-centric Spatio-Temporal Video Grounding via the Combination of Mutual Matching Network and TubeDETR

In this technical report, we represent our solution for the Human-centric Spatio-Temporal Video Grounding (HC-STVG) track of the 4th Person in Context (PIC) workshop and challenge. Our solution is built on the basis of TubeDETR and Mutual Matching Network (MMN). Specifically, TubeDETR exploits a video-text encoder and a space-time decoder to predict the starting time, the ending time and the tube of the target person. MMN detects persons in images, links them as tubes, extracts features of person tubes and the text description, and predicts the similarities between them to choose the most likely person tube as the grounding result. Our solution finally finetunes the results by combining the spatio localization of MMN and the temporal localization of TubeDETR. In the HC-STVG track of the 4th PIC challenge, our solution achieves the third place.
Simulating Visual Mechanisms by Sequential Spatial-Channel Attention for Image Quality Assessment

As a subjective concept, image quality assessment (IQA) is significantly affected by perceptual mechanisms. Two mutually influenced mechanisms, namely spatial attention and contrast sensitivity, are particularly important for IQA. This paper aims to explore a deep learning approach based on transformer for the two mechanisms. By converting contrast sensitivity to attention representation, a unified multi-head attention module is performed on spatial and channel features in transformer encoder to simulate the two mechanisms in IQA. Sequential spatial-channel self-attention is proposed to avoid expensive computation in the classical Transformer model. In addition, as image rescaling can potentially affect perceived quality, zero-padding and masking with assigning special attention weights are performed to handle arbitrary image resolutions without requiring image rescaling. The evaluation results on publicly available large-scale IQA databases have demonstrated outstanding performance and generalization of the proposed IQA model.
Cascaded Decoding and Multi-Stage Inference for Spatio-Temporal Video Grounding

Human-centric spatio-temporal video grounding (HC-STVG) is a challenging task that aims to localize the spatio-temporal tube of the target person in a video based on a natural language description. In this report, we present our approach for this challenging HC-STVG task. Specifically, based on the TubeDETR framework, we propose two cascaded decoders to decouple spatial and temporal grounding, which allows the model to capture respective favorable features for these two grounding subtasks. We also devise a multi-stage inference strategy to reason about the target in a coarse-to-fine manner and thereby produce more precise grounding results for the target. To further improve accuracy, we propose a model ensemble strategy that incorporates the results of models with better performance in spatial or temporal grounding. We validated the effectiveness of our proposed method on the HC-STVG 2.0 dataset and won second place in the HC-STVG track of the 4th Person in Context (PIC) workshop at ACM MM 2022.
Visual Signal Assessment, Analysis and Enhancement for Low-resolution or Varying-illumination Environment

More often than not, practical application scenarios call for systems to be capable of dealing with input visual signals with low resolution/quality or environmental illumination. This talk will introduce related recent research in super-resolution reconstruction, signal quality assessment, content enhancement, and person re-identification for low-resolution or varying illumination. We will also discuss possible new research attempts to advance the relevant techniques.
Fine-grained Video Captioning via Precise Key Point Positioning

In recent years, a variety of excellent dense video caption models have emerged. However, most of these models focus on global features and salient events in the video. For the makeup data set used in this competition, the video content is very similar with only slight variations. Because the model lacks the ability to focus on fine-grained features, it does not generate captions very well. Based on this, this paper proposes a key point detection algorithm for the human face and human hand to synchronize and coordinate the detection of video frame extraction, and encapsulate the detected auxiliary features into the existing features, so that the existing video subtitle system can focus on fine-grained features. In order to improve the effect of generating subtitles, we further use the TSP model to extract more efficient video features. Our model has better performance than the baseline.
Panel Discussion

With video capture devices becoming widely popular, the amount of video data generated per day has seen a rapid increase over the past few years. Browsing through hours of video data to retrieve useful information is a tedious and boring task. Video Summarization technology has played a crucial role in addressing this issue and is a well-researched topic in the multimedia community. This panel aims to bring together researchers working on relevant backgrounds to discuss emerging topics on video summarization, including recent developments, future directions, challenges, solutions, potential applications and other open problems.
In-training Restoration Models Matter

Full-Reference Image Quality Assessment (FR-IQA) metrics such as PSNR, SSIM, and LPIPS have been widely adopted for evaluating image restoration (IR) methods. However, pristine-quality images are usually not available, making inferior No-Reference Image Quality Assessment (NR-IQA) metrics seem to be the only solutions in practical applications. Fortunately, when evaluating image restoration methods, paired degraded and restoration images are generally available. Thus, this paper takes a step forward to develop a Degraded-Reference IQA (DR-IQA) model while respecting its correspondence with FR-IQA metrics. To this end, we adopt a simple encoder-decoder as DR-IQA model, and take paired degraded and restoration images as the input to predict distortion maps guided by FR-IQA metrics. More importantly, due to the diversity and continuous development of image restoration models, it is difficult to make the DR-IQA model learned based on a specific restoration model generalize well to other ones. To address this issue, we augment the DR-IQA training samples by adding the results produced by in-training restoration models. Benefiting from the diversity of training samples, our learned DR-IQA model generalizes well to unseen restoration models. We respectively test our DR-IQA models on various image restoration tasks,e.g., denoising, super-resolution, JPEG deblocking, and complicated degradations, where our method can further close the performance gap between FR-IQA metrics and the state-of-the-art NR-IQA methods. Moreover, experiments also show the effectiveness of our method in performance comparison and model selection of image restoration models without ground-truth clean images. Source code will be made publicly available.
No-Reference Quality Assessment of Stereoscopic Video Based on Deep Frequency Perception

The purpose of stereo video quality assessment (SVQA) is to easily and quickly measure the quality of stereo video, and strive to reach a consensus with human visual perception. Stereo video contains more perceptual information and involves more visual perception theory than 2D image/video,making SVQA more challenging. Aiming at the effect of distortion on the frequency domain characteristics of stereo video, a SVQA method based on frequency domain depth perception is proposed. Specifically, the frequency domain is utilized while minimizing the changes in the existing network structure to realize the in-depth exploration of the frequency domain characteristics without changing the original frame size of stereo video. Experiments are carried out on three public stereo video databases, namely NAMA3DS1-COSPAD1 database, WaterlooIVC 3D Video Phase I database, and QI-SVQA database. From the experimental results, it can be seen that the proposed method has good quality prediction ability, especially on asymmetric compressed stereo video databases.
Worst-Case Morphs: a Theoretical and a Practical Approach

Face Recognition (FR) systems have been shown to be vulnerable to morphing attacks.We examine exactly how challenging morphs can become.By showing a worst-case construction in the embedding space of an FR system and using a mapping from embedding space back to image space we generate images that show that this theoretical upper bound can be approximated if the FR system is known.The resulting morphs can also succesfully fool unseen FR systems and are useful for exploring and understanding the weaknesses of FR systems.Our method contributes to gaining more insight into the vulnerability of FR systems.
Evaluating Face Image Quality Score Fusion for Modern Deep Learning Models

Face image quality assessment algorithms attempt to estimate the utility of face images for biometric systems, typically face recognition, since the performance of these systems can be limited by the image quality. Hand-crafted quality score fusion has previously been examined for a variety of mostly factor-specific quality assessment algorithms. This paper instead examines score fusion for various recent “monolithic” quality assessment deep learning models. The evaluation methodology is based on Error-versus-Reject-Characteristic partial-Area-Under-Curve values, which are used to quantitatively rank quality assessment configurations in a face recognition context. Mean quality score fusion configurations were found to slightly improve performance on the TinyFace database, while the tested fusion types were ineffective on the LFW database.
Fusion of Face Demorphing and Deep Face Representations for Differential Morphing Attack Detection

Algorithm fusion is frequently employed to improve the accuracy of pattern recognition tasks. This particularly applies to biometrics including attack detection mechanisms. In this work, we apply a fusion of two differential morphing attack detection methods, i.e. Demorphing and Deep Face Representations. Experiments are performed in a cross-database scenario using high-quality face morphs along with realistic live captures. Obtained results reveal that a weighted sum-based score-level fusion of Demorphing and Deep Face Representations improves the morphing attack detection accuracy. With the proposed fusion, a detection equal error rate of 4.9% is achieved, compared to detection equal error rates of 5.6% and 5.8% of the best individual morphing attack detection methods, respectively.
An anomaly detection approach for backdoored neural networks: face recognition as a case study

Backdoor attacks allow an attacker to embed functionality jeopardizing proper behavior of any algorithm, machine learning or not. This hidden functionality can remain inactive for normal use of the algorithm until activated by the attacker. Given how stealthy backdoor attacks are, consequences of these backdoors could be disastrous if such networks were to be deployed for applications as critical as border or access control. In this paper, we propose a novel backdoored network detection method based on the principle of anomaly detection, involving access to the clean part of the training data and the trained network. We highlight its promising potential when considering various triggers, locations and identity pairs, without the need to make any assumptions on the nature of the backdoor and its setup. We test our method on a novel dataset of backdoored networks and report detectability results with perfect scores.
Development of an Automated Medical Information System for the Analysis of Fluorograms

This article highlights aspects of the development of a medical information system for the analysis of fluorograms using convolutional neural networks. As part of the work done, an algorithm for the operation of the module for recognizing positive and negative images, as well as the implementation of this algorithm, was obtained. The developed module is of practical value for organizing a patient’s medical profile and reduces the human factor in the process of processing medical images. It is assumed that the developed system will have the required level of process automation and will consist of several interconnected blocks, including special software, the ability to send processed data via the TCP/IP protocol to the ports of the router, as well as to confirm that the received image corresponds to a standard healthy patient, or to detect deviations. The processed version of the image is sent back to the institution where the patient is being examined.
The Secure Information Transfer Dynamic System Based on Associative Data Protection Mechanism

Steganography recently occupies serious attention to the specialty-protecting sheets. Associative steganography is focused on data protection when analyzing scenes. The analysis of scenes implies a consolidated description of images in terms of “objects – coordinates”. In contrast to the known methods of steganographic transformation, an associative approach can provide almost unconditional stability, and in comparison, with known ciphers – higher noise immunity during storage and transmission of information on open communication channels. The application of chaotic signals in modern radar and telecommunications is an actual task that can significantly extend the functionality of these systems and improve their performance. This article discusses a model of the information transmission dynamic system based on the method of switching chaotic modes. The system is designed to transmit a digital information signal hidden by the associative data protection mechanism.
The Impact of Cloudy Weather on the Operation of Photovoltaic Power Plant in the Northern Part of the Russian Far East

This paper is devoted to studying the working characteristics of photovoltaic plants (PV) in different cloud conditions. A technique for determining the electrical energy efficiency (PV) in different clouds types is described, with graphical explanations of the experimental work procedure. The authors of the paper obtained control parameters for changing the value of the energy generated in a solar power plant under different clouds types, which can be used in operational process modeling and engineering calculations for operating solar power plants. It has been determined that the operation (PV) under different clouds types where the generating capacity of the plant varies in the range (8-95%) compared to the clear sky. The purpose of this research is to use the obtained indicators of changes in the generating capacity of the plant for different clouds types in developing a methodology to assess the clouds effectand their types on the throughput of solar radiation falling on the surface of solar panels.
The Types of Filtering Images with Cloudy Weather

In the article we have investigated and studied the available types of filtration. The scientific novelty of this study is the proposed method of the highest quality image filtering in rainy weather. Based on the results of this study, we implemented computer programs for studying filters, and conducted a comparative analysis of the results of using various nonlinear methods to clean images from noise. We evaluated the possibilities of using these filters during rain. The main operations of mathematical morphology are build up, erosion, closure and opening. These names reflect the essence of the operations. So, the build-up increases the image area. The erosion makes it smaller. The closure operation allows you to close the internal openings of the area and eliminate the bays along the border of the area. The opening operation helps to get rid of small fragments protruding outside the area near its border.
Automation of Program Code Analysis Using Machine Learning Methods

The paper presents a description of the developed approach and service for analyzing source code in Python. The service allows you to reduce the time spent on the code review process by partially automating this process. After publishing a changeset to a remote Git repository, the service starts checking the changeset and publishes reports of bugs and duplicates found in changeset comment format. A neural network with an original architecture is used for automatic code reviews. PyTorch, Scikit-learn and Transformers libraries are also used. The testing performed did not reveal any errors that affect the operation. All the main functions of the system are performed correctly.
Development of a Chatbot for the Subject Area Train Schedule for Telegram

In this article, using the Twin question-and-answer system, was developed and tested a chatbot for the subject area “Train Schedule” for Telegram. This area is quite promising, because it greatly simplifies the process of communication with customers. During buying tickets, users often have a lot of questions, but not all the information is available on the site. Also, the search process takes a long time, and a person would have to call the train station, where also will be stand in line. This bot has been tested in a variety of situations. The results of testing showed that there are some areas, where the bot responds incorrectly. After analyzing the errors, it was discovered that it was possible to correct these errors and achieve better results.
Implementation Method of Rendezvous Process of Unmanned Aerial Vehicle Landing on the Mobile Unmanned Vehicle

The implementation process of the UAV landing on moving UV is the most difficult during the operation of UV-UAV complex. In this paper, the method of realizing the rendezvous process of UAV and UV in complex interference environment is considered. Researched unmanned aerial vehicles - tiltrotor with an unmanned vehicle for landing on a platform in the body of a moving vehicle has been studied. Proposed solution method for unmanned aerial vehicles to approach unmanned vehicles using homing technology. Presented homing methods and operating principles of various homing algorithms. Thereby, the positioning accuracy of the UAV landing is improved, and the safety during the UAV and UV round process is guaranteed.
Implementation of brightness and color equalization methods to create a smooth panoramic image on a real ADAS platform

This paper proposes and implements brightness and color equalization (BCE) methods of images obtained from different cameras on a real advanced driver assistance system (ADAS) platform. In particular, four different methods for BCE that are a combination of a basic linear transformation (BLT) and gamma correction are implemented and tested. In addition to BCE, blending operations based on improved gradual fusion were also carried out to obtain a smooth transition of brightness and color components in the final panoramic image at the overlapping region between the input images. The resulting images were tested subjectively to determine the most preferred method for BCE out of four methods. The developed solution for BCE was implemented on the ADAS ALPHA development board.
Review and Analysis of Information Security Issues in Data Exchange and Management of a Virtualized Environment of Remote Data Processing Centers

The article deals with issues related to information security when exchanging data and managing a virtualized environment of remote data processing centers. As of today, it has become clear, that many enterprises have switched to the use of virtualization and cloud computing systems. Therefore, there appears an important task to design and implement a reliable and efficient information security system. The article contains an analysis of the mechanisms for building a data center, and models for providing services. It also contains consideration of the existing attacks on VI (virtual infrastructure) and measures proposed to ensure a high level of system security. The purpose of the article is to study and analyze security issues in information exchange and management of a virtualized environment of remote data centers. According to the goal set in the article, there was conducted a study of the problems of information security in the exchange of data and management of the virtualization environment of remote data centers. This study has resulted in a given list of measures and recommendations aimed at the increase of the security level of information protection systems in the data center. The practical significance of the results obtained in the article lies in the formation of recommendations for setting up the data center information security system. The conclusion of the article contains inferences drawn on the results of the study.
Simulation of Structured Light 3D Scanning using Blender

Many different optical setups and patterns are proposed for structured light 3D surface scanning. It is difficult and time consuming to experimentally assess all imaging options and to select the best one for a particular application. An efficient way of assessing imaging options is simulation of 3D scanning on a computer. We describe an efficient and low cost approach to simulating structured light scanning using Blender. Blender is an open source 3D computer graphics program primarily intended for 3D drawing and animation also supporting Phyton scripting to extend its capabilities. We describe how to simulate a simple scanning setup comprised of one camera, one projector, and one object. We also provide detailed description how to set both intrinsic and extrinsic parameters of projector and camera and discuss how to automate the simulation. To assess the correctness of the proposed simulation we simulate a real-world 3D surface scanner and we compare simulation results to the real-world results. We also perform experiments of scanning calibration objects. The proposed simulation method is correct and can be use to assess imaging geometry, to select structured light patterns, or to generate data for novel pattern decoders which use deep learning.