Reviewerly/exHarmony huggingface dataset:
MobileViT-GAN: A Generative Model for Low Bitrate Image Coding

Effective image coding techniques are crucial for digital image storage and transmission. Traditional methods struggle to maintain high visual quality at low bitrates. In this paper, we present MobileViT-GAN, a novel generative adversarial network (GAN) architecture for low bitrate image compression. We propose a lightweight transformer-based discriminator to improve coding performance, compared to convolutional neural network-based discriminators. Additionally, we introduce a smoothness loss function to mitigate artifacts in decoded images, further improving visual quality in low bitrate image coding. We evaluate our proposed method against traditional and state-of-the-art GAN-based image compression techniques, showcasing its superiority in terms of compression ratio and decoded image quality.
Lifecycle Management of Federated Learning Artifacts in Industrial Applications

In industrial settings, traditional centralized ap-proaches for training AI models can be insufficient due to limited training data. Industrial Federated Learning (IFL) offers a promising solution by enabling collaborative training across multiple industrial devices, while keeping data on-premises. In this paper, we propose a novel approach for supporting the development, deployment, integration and execution of IFL solutions. The proposed method provides a lifecycle management of FL artifacts and supports FL as a Service (FlaaS). This enables the extensibility and customizability of FL-based edge applications in industrial settings. Additionally, we introduce a federated clustering algorithm that we have integrated into a condition monitoring app running on client locations to evaluate the proposed lifecycle management. We run two scenarios with four and 33 clients using real-world time series data from industrial pumps. Our results show the applicability of the implemented lifecycle management and demonstrates that privacy-preserving approaches compete well with privacy-disclosing ones.
Pixel-Unshuffled Multi-level Feature Map Compression for FCVCM

The feature compression process for machine tasks involves several steps: feeding a video into the task network, extracting intermediate feature maps, compressing these maps into a bitstream on the client side, transmitting the bitstream to a resource-rich server, decoding it, and ultimately completing the specific machine task. In this paper, we present a multi-level feature compression method designed for machine tasks. We introduce an efficient and effective feature reshaping and merging module within the PCA-based feature coding scheme. This module utilizes pixel-unshuffled operations to reshape the multi-level features, merges them into a single map, and then performs a transformation. Our proposed method achieves a BD-rate gain of 49.69% and 66.3% in comparison to the previous computational low cost PCA-based feature coding method for object detection and instance segmentation tasks, respectively.
Towards Efficient Learned Image Coding for Machines via Saliency-Driven Rate Allocation

Recently, efficient image coding for machines method is widely required under machine vision task-oriented coding scenarios. To achieve higher task accuracy under a certain bitrate constraint, existing solutions take advantage of cutting-edge learned image compression methods and cascade the compression and specific task model straightly to carry out the joint optimization, which lacks adaptability and flexibility to handling machine vision saliency and non-saliency regions, thus obtaining inferior performance. In this paper, we propose a learned image compression model with a saliency-driven rate allocation strategy for machine vision. Specifically, an auxiliary network is introduced to provide reference latent first which contains full image reconstruction required information. And then, a machine vision saliency detector is applied to generate the adjustment factor to achieve rate allocation in the latent domain. In addition, a spatial dynamic convolution module is designed to facilitate the compressor adaptively processing the different spatial areas. Finally, the multi-distortion constraint is utilized to achieve an end-to-end optimization. Experimental results show that the proposed method achieves -77.48% coding gains over VVC using Mask R-CNN as the back-end task network on the Cityscapes dataset. Moreover, the reconstructed image maintains considerable visual fidelity, making it possible both for human and machine consumption.
Distributed Routing Algorithms for Concurrent Execution of Transactions in PCNs

Payment Channel Networks (PCNs) are an alternative to improve the scalability of a blockchain network. The network size of Bitcoin PCN is increasing rapidly; in the past three months, the number of nodes has more than doubled. With the increase in the network size, transactions on the network will also increase. However, the existing distributed routing algorithms cannot efficiently schedule concurrent transactions due to their static nature. We propose two algorithms, maxECW and maxSCL, which can handle concurrent transactions more efficiently. Our algorithms consider channel weights and introduce the concept of rebalancing to avoid the saturation of the directional capacity of a channel. We have also developed a simulator, DRLN sim, to compare our proposed algorithms with the existing ones. The routing algorithms are evaluated on the simulator by varying relevant parameters. On average, our proposed algorithms performed 50% more efficiently than existing algorithms in handling concurrent transactions.
Object Detection Acceleration Method by Improving Execution Efficiency of Edge AI Device

In recent years, there has been a growing need to perform object detection at the edge. Since the edge environment has tight physical constraints, the efficient use of AI devices is a key challenge to executing object detection at high throughput. In this paper, we propose an object detection acceleration method that uses two types of one-stage detectors in combination. After detecting object candidates by a lightweight detector, the method generates aggregated images by combining the candidate images and executes the second, more accurate detector on the aggre-gated images to improve the execution efficiency of AI devices. Our evaluations confirmed that the proposed method could improve object detection throughput by up to eight times for a license plate detection task with almost no accuracy degradation. We also conducted evaluations with a car detection task and a pose estimation task and confirmed the broad applicability of the proposed method.
Unified Measures for the Rate-Distortion-Latency Trade-off

In today’s digital age, multimedia content is omnipresent, and the demand for efficient compression techniques is ever-increasing. In particular, the successful delivery of services based on video transmission largely depends on achieving the lowest latency values. One solution has been to use extrapolation for latency compensation in video transmission that allows to reduce the latency by an arbitrary amount. Nevertheless, this latency reduction comes at the cost of an increased distortion of the displayed images, since they are based on temporal extrapolation. Latency can also be traded with coding rate. This paper introduces ELR-PSNR and EPR-Latency as unified metrics to assess the three-way trade-off between rate, distortion, and latency simultaneously.
An Advanced Multi-Scale Feature Compression using Selective Learning Strategy for Video Coding for Machines

Machine vision-based applications have witnessed widespread adoption in diverse fields. Efficiently processing and compressing the vast amounts of video data collected by machines is crucial for these applications. To address this need, the Moving Picture Experts Group (MPEG) is developing a new' video coding standard known as Video Coding for Machines (VCM), specifically optimized for video consumed by machines in vision applications. This paper proposes an advanced multi-scale feature compression (advMSFC) method with a selective learning strategy (SLS) for the Feature Compression for VCM (FCVCM). By applying the SLS, the proposed method converts multi-scale features into a single-scale feature arranged based on channel-wise importance, enabling adaptive feature channel truncation based on the QP. The truncated feature is efficiently compressed using the latest video codec, Versatile Video Coding (VVC). The proposed method outperforms the VCM feature anchor in instance segmentation, object detection, and object tracking tasks, achieving significant Bjontegaard delta-rate (BD-rate) gains. The adaptability of our method using a single trained model for various QPs shows promise for efficient feature compression in machine vision applications.
STMotionFormer: Language Guided 3D Human Motion Synthesis with Spatial Temporal Transformer

Inspired by the Transformer's adaptive capability in natural language processing and computer vision domains, and supported by CLIP's strong semantic prior knowledge, we propose a novel generative model named STMotionFormer based on improved Transformer block and CLIP for language-guided 3D skeleton-based human motion synthesis. The skeleton sequence, as a kind of spatial temporal dynamic data, not only contains temporal information but also consists of a natural graph structure by different body parts. We focus on this, propose Temporal Transformer block (T-Former) applies attention mechanism along the temporal dimension to capture the long-term relationship, and propose spatial Transformer block (S-Former) applies attention mechanism in the spatial dimension to aggregate and update joints' spatial features. Therefore, our approach can automatically explore both spatial and temporal information, which can generate skeleton sequence with reasonable graph structure as well as global time relationship. Moreover, powerful semantic prior knowledge of CLIP are injected into our motion-language joint manifold. We evaluate our method on the public KIT motion language dataset that contains manually annotated 3D pose sequences. Experimental results show that our model outperforms the state-of-the-art in terms of APE and AVE respectively. Qualitative visualization results indicate that our model can generate human motions that are more compatible with semantic information even than the GroundTruth.
Mask-KLT: Sub-pixel Accurate Directional Motion Estimation by Stripe Masking

To diagnose the safety or monitor the health of aging infrastructures and factory facilities, a way to measure precise vibration or displacement is required. In this paper, we present a sub-pixel accurate tracking method of directional motion of interest, which we name Mask-KLT by inheriting the long-standing Kanade–Lucas–Tomasi tracker. Our simple method can estimate directional displacements of an object of interest along any single axis and is also robust to non-corner regions lacking good features to track. Our algorithm can be easily implemented by simple modification of the KLT tracker: by pre-processing an input image with a stripe pattern masking. We evaluate our proposed method with sub-pixel accuracy on a synthetic dataset. Our Mask-KLT dramatically increases the success rate of tracking and decreases displacement errors compared to Vanilla KLT. Moreover, we demonstrate the effectiveness of our method in a real-world experimental setup: a rotating machine in a laboratory environment.
Sophisticated Swarm Reinforcement Learning by Incorporating Inverse Reinforcement Learning

In the last decades, the reinforcement learning method has attracted a great deal of attention and many studies have been done. However, the method is basically a trial-and-error scheme and it takes much computational time to acquire optimal strategies. Furthermore, optimal strategies may not be obtained for large and complicated problems with many states. To resolve these problems we have proposed the swarm reinforcement learning method, which is developed inspired by the multi-point search optimization methods. In this paper, we propose a sophisticated swarm reinforcement learning method incorporating inverse reinforcement learning, which can improve learning speed and obtain better solutions. The proposed method is developed especially for the partially observable Markov decision processes (POMDPs). We evaluate the proposed method through experiments and compare the results with those of the conventional swarm reinforcement learning, Q-learning and Hierarchical Q-learning (HQ-learning), It is confirmed that the proposed method makes it possible to obtain better solutions with less learning time than the existing methods, especially for the problem in POMDPs.
Enhancing the Discriminative Ability for Multi-Label Classification by Handling Data Imbalance

In computer vision, long-tailed multi-label visual recognition is a challenging problem due to the imbalance between classes and the recognition of rare classes. Previous methods for resolving data imbalance mostly originate from single-label classification, which can be obstructed by the label co-occurrence issue, and they attempt to minimize or compensate for it. In this paper, we propose a novel tail class priority sampling method for long-tailed multi-label classification that untangles both issues. Our method samples tail class more often and earlier in order to make the model learn tail classes before it has bias toward common classes. Due to label co-occurrence, other classes will be spontaneously learned in same iteration, ensuring a balanced representation of the head and medium classes. To further enhance the recognition performance, we modify to a bilateral structure that samples both original and proposed sampling distribution to better represent the tail classes. We evaluate our proposed method on two widely used datasets in long-tailed version, COCO-LT and VOC-LT, and compare it with previous methods. The experimental results show that our method achieves a new state-of-the-art performance for tail classes on both datasets. Our method is applicable in various real-world scenarios, making rare class recognition achievable, and can be easily incorporated into conventional recognition frameworks.
Enlarged Large Margin Loss for Imbalanced Classification

We propose a novel loss function for imbalanced classification. LDAM loss, which minimizes a margin-based generalization bound, is widely utilized for class-imbalanced image classification. Although, by using LDAM loss, it is possible to obtain large margins for the minority classes and small margins for the majority classes, the relevance to a large margin, which is included in the original softmax cross entropy loss, is not be clarified yet. In this study, we reconvert the formula of LDAM loss using the concept of the large margin softmax cross entropy loss based on the softplus function and confirm that LDAM loss includes a wider large margin than softmax cross entropy loss. Furthermore, we propose a novel Enlarged Large Margin (ELM) loss, which can further widen the large margin of LDAM loss. ELM loss utilizes the large margin for the maximum logit of the incorrect class in addition to the basic margin used in LDAM loss. Through experiments conducted on imbalanced CIFAR datasets and large-scale datasets with long-tailed distribution, we confirmed that classification accuracy was much improved compared with LDAM loss and conventional losses for imbalanced classification.
Perceptual Quality Assessment for Video Frame Interpolation

The quality of frames is significant for both research and application of video frame interpolation (VFI). In recent VFI studies, the methods of full-reference image quality assessment have generally been used to evaluate the quality of VFI frames. However, high frame rate reference videos, necessities for the full-reference methods, are difficult to obtain in most applications of VFI. To evaluate the quality of VFI frames without reference videos, a no-reference perceptual quality assessment method is proposed in this paper. This method is more compatible with VFI application and the evaluation scores from it are consistent with human subjective opinions. A new quality assessment dataset for VFI was constructed through subjective experiments firstly, to assess the opinion scores of interpolated frames. The dataset was created from triplets of frames extracted from high-quality videos using 9 state-of-the-art VFI algorithms. The proposed method evaluates the perceptual coherence of frames incorporating the original pair of VFI inputs. Specifically, the method applies a triplet network architecture, including three parallel feature pipelines, to extract the deep perceptual features of the interpolated frame as well as the original pair of frames. Coherence similarities of the two-way parallel features are jointly calculated and optimized as a perceptual metric. In the experiments, both full-reference and no-reference quality assessment methods were tested on the new quality dataset. The results show that the proposed method achieves the best performance among all compared quality assessment methods on the dataset.
Analysis of Needs and Expectations in Business Education of Engineering Students

The article is devoted to the analysis of local features, needs and expectations in modern methods of business education of students of engineering and other specialties. Significant needs for business education of students of both economic and engineering specialties have been identified. The analysis of students' needs and expectations was based on a survey of 2,294 students from Ukraine and Azerbaijan during February-May 2023. Educational activities on business education are planned to be implemented within the ERAZMUS+ project regarding the implementation of a network of international virtual youth business centers in Ukraine, Azerbaijan and other countries, which is implemented by the Kyiv National University of Technology and Design and promotes the creation of an international network of youth virtual business hubs for students of universities, technical schools and schools. Most students have a need for business education programs, both synchronous and asynchronous. Based on participation in international youth virtual mobility programs, students want to develop and improve their communication skills: the level of foreign language proficiency, communicate with foreign peers, including through international virtual hubs. Students also want to develop business competencies, skills to create their own business, skills to create and implement startups.
Monocular vision-based correction method for the position of robot screw nail

In the process of satellite assembly, there are errors in the installation and positioning of the cabin board and the frame, which leads to the deviation between the position of the screw screwed by the teaching robot and the actual position of the screw hole. In order to solve this problem, a method of robot screw position and orientation correction based on monocular vision is proposed. The Eye-in-Hand structure was used to capture images online by monocular camera. After filtering and denoising the images, the Canny operator was used to extract the feature contour of nail holes. In view of the fracture of the nail hole feature contour, a fracture contour similarity splicing algorithm was proposed, so as to retain the original information of the nail hole more comprehensively. The image was upgraded to the sub-pixel level, and the position of nail holes in the image was extracted with high precision through sub-pixel edge detection. The homologous relation between the actual image and the desired image is calculated to estimate the pose of the robot when taking the desired image. According to the prior relationship between the desired pose when the robot takes a picture and the desired pose when the robot tightens the screws, the actual robot's tightening screw position was corrected. The deviation threshold of the image and the error threshold of the robot kinematics were set, and then the correction times of the position of the screw tightened by the robot were constrained. Finally, the experimental results show that the proposed method can meet the accuracy requirements of robot screw nails in the satellite assembly process.
Modelling and Simulation of Autonomous Decision - making Process Based on Proximal Policy Optimization

The autonomous decision-making problem in the mixed process of medium-range and close-range confrontation stages of agents is studied in this paper, utilizing reinforcement learning theory. The complexity and intensity of the environment resulting from the integration of information technology and system-of-systems (SoS) support in the agent confrontation scenario, as well as the oversimplification of scenarios in traditional research on autonomous decision-making of agents, are taken into account. Firstly, a simulation platform is constructed based on the Agent-Based Modeling and Simulation (ABMS). Then, the state space, action space, and reward function are designed to adapt to the multi-stage confrontation scenario for agents. Finally, an autonomous decision-making method based on Proximal Policy Optimization (PPO) algorithm for agent confrontation is proposed. The experimental results demonstrate that the proposed decision-making algorithm can learn reasonable confrontation tactics in multi-stage confrontation scenarios, verifying the feasibility of studying the problem based on the above modeling and simulation framework.
HLRS: A Deep Reinforcement Learning-Based Hero Recommendation System for MOBA Games

Multiplayer Online Battle Arena (MOBA) games have gained immense popularity in recent years, with games like Dota2, League of Legends (LoL), and Honor of Kings (HoK) being the most popular. In a typical MOBA game, two teams consisting of 10 players battle against each other in two phases: hero selection and hero battle phase. Choosing a suitable virtual game character from all the available heroes during the hero selection phase is a daunting task due to the large number of current heroes, their complex relationships, and player interests. Therefore, we propose an innovative hero recommendation system called HLRS, which acts as a coach to guide players in selecting appropriate heroes during the selection phase. To verify the effectiveness of our model, we used HoK as a test platform and trained the model using its game data. We conducted numerous experiments, and the results showed that our model was more effective than other recommendation schemes. It analyzed various aspects to recommend the most suitable hero characters for players.
Relation Extraction with Knowledge-Enhanced Prompt-Tuning on Multimodal Knowledge Graph

Recently, Multimodal Knowledge Graphs (MKGs) with visual and textual factual knowledge have been widely used in tasks such as knowledge question answering, recommender systems, and entity disambiguation. Since most of the current MKGs still have defects, a multimodal knowledge graph completion technology is proposed, and multimodal relation extraction (MRE) is one of the basic processes. However, visual objects with high object classification scores are usually selected in previous tasks, which may result in the addition of noise from objects that are either irrelevant or redundant, which can adversely affect multimodal relationship extraction. For this reason, in this paper, we propose a Relation Extraction with Knowledge-enhanced Prompt-tuning modal on multimodal knowledge graph (REKP) to address these issues. Specifically, we inject potential knowledge from relational labels into the prompt construction of answer words and optimize their representation with structured constraints. A Transformer architecture with cross-modal attention is then used to fuse the visual and textual representations. We conduct extensive experiments to verify that our REKP model can achieve SOTA performance on the MNRE dataset with multimodal relational extraction.