Reviewerly/exHarmony huggingface dataset:
Improved SlowFast Network with Spatial–Temporal Attention Module for Action Recognition

With the constructing of large volume video dataset and the rapid development of machine vision technology, action recognition in videos has become a hot topic in many applications. The appearance and tempo of human actions is variant in spatial and temporal space. It’s necessary to focus on the detailed descriptions of the fast and slow variations in an action category. We embed the spatial–temporal attention module into the SlowFast networks, to increase the describing ability of fast and slow motion changes. The accuracy of the proposed method is effectively improved on UCF-101 and HMDB-51 datasets. Experiments validate the effectiveness of the networks with embedded spatial–temporal attention module for discriminating variant motion tempos. The proposed method is able to capture more detailed description of action categories from the slow and fast pathways and present a more semantic recognition result.
SRPose: Low-Resolution Human Pose Estimation with Super-Resolution

2D human pose estimation from given images has been an activate research area in computer vision. Existing methods based on deep learning rely on high-resolution input, which is not always available in many scenarios. To address the issues, a novel algorithm called Super-Resolved Pose estimation(SRPose) is proposed in this paper, which is composed of a super-resolution sub-network(SRN) and a following human pose estimation sub-network(HPEN). The SRN equipped with global residual learning and position-preserving block constructs a HR version from a LR input and then HPEN perform pose estimation. The whole SRPose is optimized with a unified loss in end-to-ento-en. Comprehensive experiments on public benchmarks verify the effectiveness and generalization of the proposed SRPose under the condition of the LR input.
Inactive Region Filling Method for Efficient Compression Using Reinforcement Learning

Based on the massive advancements in modern hardware technologies, beyond high-resolution images, immersive and interactive videos have become an important next-generation technology. Immersive video includes 3D information having a huge number of bits. Thus, a special coding standard is required to compress immersive video e.g., 360 and point cloud videos. To compress immersive videos, the 3D information is projected into 2D space which creates several empty spaces called inactive regions. Even though the codec model can efficiently compress the immersive video, the inactive regions excessively degrade the overall performance. Recently, several model-based methods have suggested filling the inactive regions to prevent performance degradation. In this paper, we propose a novel approach to fill the inactive region using deep reinforcement learning. We perform block-wise inactive region filling in which each block has an agent, and overall pixel values are controlled by using a control point-based block-wise selective filtering method. The control point controls the pixel intensity to generate rough patterns and reduce computational complexity. In addition, each control point and the active region (attribute or pixel information) are propagated to the inactive region by considering their own information and neighboring information. Besides, each agent can learn the information of neighboring blocks with a modified rate-distortion (R-D) cost reward. We evaluate the proposed method on the sphere segmented projection (SSP), point cloud compression (PCC) attribute, and PCC geometry video formats. The proposed method achieves an average bitrate reduction of -10.9% for the SSP, -9.1% for the PCC-attribute, and -11.8% for the PCC-geometry, which are much better than the conventional method.
DRMNet: A Multi-Task Detection Model Based on Image Processing for Autonomous Driving Scenarios

To improve the efficiency and accuracy of autonomous driving vehicles' perception of the external environment, a multi-task detection model DRMNet (Dual-resolution Multi-task Network) is proposed that can be applied to autonomous driving scenarios, which can simultaneously complete the tasks of vehicle detection, lane detection, and drivable area detection. Firstly, given the loss of feature information by multiple downsampling in the backbone feature network, which affects the detection accuracy, the backbone of the model is designed as a two-pathway structure, which is used to extract shallow detail information and deep semantic information, respectively. Secondly, a multi-scale feature fusion module (MFFM) is designed to fuse the extracted shallow detail and deep semantic information. Then, different detection branches are designed according to the different characteristics of each detection task. Finally, Experiments on the BDD100K show the performance of DRMNet in three detection tasks: The recall and mAP of vehicle detection are 93.9% and 80.0% respectively. The accuracy of lane detection is 76.3%. The mIoU of drivable area detection is 92.2%. It is superior to the existing multi-task algorithm model, and the model has good generalization ability through actual scene experiments.
HSM-QA: Question Answering System Based on Hierarchical Semantic Matching

In recent years, Question Answering (QA) systems have gained popularity as a means of acquiring knowledge. However, the prevalent approach of matching question-answer pairs still suffers from low precision and efficiency due to the inherent ambiguity of natural language descriptions. To address these issues, we propose a novel QA approach based on hierarchical semantic matching, termed HSM-QA. Specifically, HSM-QA is decomposed into two main steps, i.e., query-question and query-answer matchings, respectively. For query-question matching, a Siamese network is applied to calculate the similarity between query-question pairs, which recalls the most similar questions and their corresponding answers as candidates. In terms of query-answer matching, we adopt the idea of the pairwise algorithm and propose a single-stream structure to calculate the relevance between query and answer, based on which the best-matching candidates are ranked and returned. After training, these two steps are combined as an efficient QA scheme for different languages, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., English and Chinese. Furthermore, to address the lack of Chinese QA datasets, we collect a massive amount of text data from Chinese social media and generate a new dataset via a pre-trained language model. Extensive experiments are conducted on six QA datasets to validate our HSM-QA. The experimental results demonstrate the superior performance and efficiency of our method than a set of compared methods.
Robust Multi-Object Tracking With Local Appearance and Stable Motion Models

Multi-object tracking (MOT) has been steadily studied for video understanding in computer vision. However, existing MOT frameworks usually employ straightforward appearance or motion models and may struggle in dynamic environments with similar appearance and complex motion. In this paper, we present a robust MOT framework with local appearance and stable motion models to overcome these two hindrances. The framework incorporates object and local part detectors, a feature extractor, a keypoint extractor, and a data association method. For the data association, we utilize five types of similarity metrics and a cascaded matching strategy. The local appearance model is suggested to be used additionally with global appearance features of full bounding boxes to obtain discriminative features even for objects with a similar appearance. At the same time, the stable motion model considers the core of the body as the central point of the object and subdivides the body using a novel 12-tuple Kalman state vector to analyze complex motion. As a result, our new tracker achieves state-of-the-art performance on the DanceTrack test set, surpassing all other listed tracking systems in terms of both detection and tracking quality metrics, including HOTA, DetA, AssA, and MOTA. The source code is available at https://github.com/Jubi-Hwang/Robust-MOT-with-Local-Appearance-and-Stable-Motion-Models.
Case Study on the Terms Novice Programmers Use to Describe Code Snippets in Java

Contribution: Most work on languages in computing education currently focuses on non-native speakers. In contrast, to the best of the authors’ knowledge, this article is the first response to the call for research on terms that takes into account the terms used by novices in their language.Background: Terms are key factors in communication, thinking, and belonging to a community, but questions about terms and their use by novices are listed as research calls that have not yet been answered for novices. Terms can be used to identify misconceptions about programming languages (PLs) and conceptions, which is an ongoing trend in computing education research.Research Questions: RQ1: What terms do novices use to describe code snippets in Java? RQ2: To what extent and in what way do the terms indicate PL misconceptions or conceptions?Methodology: An inductive-dominant qualitative content analysis (QCA) was conducted to examine the terms used by novices. The data consisted of more than 1800 free-text responses from 123 undergraduate CS1 students from the fall 2020/2021 semester.Findings: In general, novices use technical language appropriately in their terms when describing code but individual responses and terms revealed a wide diversity. The terms reflect both undescribed PL misconceptions and conceptions, confirming previous research in this area.
Parameters Transfer Framework for Multi-domain Fake News Detection

Recently, various BERT-based architectures for fake news detection have been proposed. In many cases, these methods work well on various benchmark datasets. However, the performance quickly deteriorates when the models are tested with samples coming from distributions which essentially differ from the ones the model has been trained on. To overcome this obstacle, in this paper, we have proposed a framework for the model’s parameter transfer in order to redefine fake news detection as a multi-domain classification problem. The proposed solution is intended to possibly leverage different elements of the classification model’s configuration to maximise global performance. The research presented in this paper utilises six publicly available, open benchmark datasets, presents the motivation, the architecture and the technical details of the proposed solution, and reports the achieved results with an array of well-known evaluation metrics. The presented results are promising and open the space for further research.
Real-Time Stereo Matching Network Based on 3D Channel and Disparity Attention for Edge Devices Toward Autonomous Driving

Stereo matching is an important component technology that constitutes the 3D perception capability of autonomous vehicles. On resource-constrained edge devices, it is very important to compute in real-time with very low time. However, most stereo matching networks focus on generating disparity maps on high-end GPUs, which do not meet the real-time requirements on edge devices. To solve this problem, a new stereo matching network is proposed in this paper to achieve real-time stereo matching on edge devices. The proposed network greatly improves the inference speed by constructing a low-resolution feature extractor, and by using multi-stage residual methods for stereo matching. In particular, we propose a method that combines the group-wise L1 distance & the group-wise correlation cost volume and an effective attention-based 3D cost aggregation method. Our network achieves a good balance between speed and accuracy on the KITTI 2012 and KITTI 2015 datasets. The proposed network achieves 2.77% and 3.44% accuracy (D1-all) on KITTI 2012 and KITTI 2015, respectively. With TensorRT, the proposed network achieves 31.8 FPS and outperforms the real-time results of most state-of-the-art networks on NVIDIA Jetson Nano edge devices.
Exploiting Sequence Analysis for Accurate Light-Field Depth Estimation

Depth estimation for light field (LF) images is the cornerstone of many applications of light field cameras, such as 3D reconstruction, defects inspection, face liveness detection, and so forth. In recent years, convolutional neural network (CNN) has dominated the primary workhorse for depth estimation. However, the interpretability of the network and the accuracy of the depth estimation results still need to be improved. This paper uses the conditional random field (CRF) theory to explain and model the LF depth estimation. Further, from the perspective of sequence analysis, we extract the sequence features of epipolar plane image (EPI) patches with recurrent neural network (RNN) and serve as the unary term of the energy function in the CRF. Then, a unified neural network (called as LFRNN) is designed to solve the CRF and get the depth map. Our LFRNN builds upon two-stage architecture, involving a local depth estimation and a depth refinement. In the first part, we design an RNN to analyze the vector sequences in EPI patches and obtain local depth values. There are two thinking behind the design of this part. The first is the general principle that the slope of the straight line in the EPI is inversely proportional to the depth; the second is our unique observation that those straight lines are distributed in vector sequences. In the second part, continuous CRF is used to optimize the output of the first part. We train LFRNN on a synthetic LF dataset and test it on both synthetic and real-world LF datasets. Quantitative and qualitative results validate the superior performance of our LFRNN over the state-of-the-art methods.
Dynamic Feature-Memory Transformer Network for RGBT Tracking

RGBT object tracking is an important research topic due to the complementary features of visible and thermal infrared images. However, the established fast-tracking Siamese series RGBT trackers do not fully exploit the predicted results and global information, which leads to poor tracking performance in challenging scenarios such as target object deformation and fast movement. To address these challenges, we propose a dynamic feature-memory transformer RGBT tracking framework in this article. Precisely, we extract the features of visible and infrared image pairs separately using convolutional networks. We then fuse the complementary semantic information using our proposed complementary semantic fusion module, which consists of a group convolution, a channel attention structure, a flattening operation, and channel reduction. Next, we concatenate the dynamic memory feature extracted from the predicted frame pair, the template pair feature, and the search region pair feature. Then input concatenated features into transformer encoder-decoder modules to acquire the global information and further fuse the RGBT features. Finally, we predict the object state using the tracking head while deciding whether to update the dynamic memory feature using the reliability estimator. We conducted extensive experiments on the widely used RGBT234 and LasHeR datasets to evaluate the proposed framework’s performance. The experimental results show that our method outperforms state-of-the-art RGBT trackers, demonstrating the effectiveness of our approach. Overall, our proposed framework leverages visible and infrared images and incorporates global information and dynamic memory features, offering a promising solution to the challenges in RGBT object tracking. Code will be open sourced at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ELOESZHANG/DFMTNet</uri>
中小製造業の経営層を対象としたデジタルものづくりのリカレント工学教育

In this paper, we will introduce our framework of recurrent engineering education. The goal of this framework is for the executives, who are the key to business transformation in their companies, to understand system integration related to digital transformation (DX) and Industry 4.0 and optimize their company as a whole by looking at the entire business. Through industry-academia-government collaboration, we (1) created a curriculum, (2) opened a business school course, and (3) held a growth strategy workshop. This paper will also introduce how small and medium-sized manufacturing companies developed their own growth strategies and started working on DX.
Concave-Hull Induced Graph-Gain for Fast and Robust Robotic Exploration

Existing RRT-based exploration methods often suffer from interruptions in the exploration process due to the inability to detect all frontiers of the drivable area in the mapping map. These methods cannot detect all frontiers because the RRT expansion is disturbed by factors such as RRT preset parameters, sliding window constraints, complex external environment, etc, and thus cannot completely cover the drivable area within a limited time. We address this problem by redefining exploration frontiers, designing a novel exploration gain, and constructing minimum RRT search spaces. Our method is evaluated against the existing state-of-the-art RRT-based methods in simulated benchmarks and outdoor environments. The results show that our method is more robust to the above factors while reducing computational cost. Our method is made open source to benefit the community.
Blockchain-Based Sustainable Retail Loyalty Program

This paper addresses the issue of sustainability in retail sales, where the composition of the products themselves, their production and packaging process, and their transport to the point of sale play an important role. We propose a solution to apply a unified system of environmental product rating in different retail chains where trust is ensured through a distributed ledger. The system allows fair and transparent rewards for customers who purchase eco-friendly products while also maintaining their privacy. The prototype of the decentralized application for the uniform loyalty system based on the Algorand blockchain has shown promising results in terms of throughput and cost efficiency. We believe that the prototype has the potential to revolutionize loyalty systems and promote sustainable practices. This study provides key insights into how to interconnect retailers and customers into a uniform reward program, how to motivate customers to purchase sustainable and eco-friendly goods, and how retailers can manage the amount of reward for the given Eco-score categories.
Cognitive Ergonomics of the Integrated Collaborative, Interactive E-Learning and E-Training- upon Innovative Networks Empowered with AI and Big Data Analytics and Computing- of the Remote E-Multidisciplinary Team Board in Breast Cancer Decision Making and Treatment

Background and purpose: The project refers to the effect that interactive E-Learning among expert participants in our innovative Remote E-Multidisciplinary Board for Breast Cancer (PROMETHEUS II pn:1009078, 34931,34932,34933) is being integrated with the Hellenic Hyper-computer ARIS for Big Data analytics and computing. Design/methodology/Results: Integrated analysis of two research projects results showed that the E-Multidisciplinary Board in Breast Cancer may also be the substance of Innovative Networks for Interactive Collaborative E-Learning and Training among experts, residents and students for further and deep learning of the cognitive ergonomics of the E-Multidisciplinary Board incorporating AI and Big Data Analytics and Computing to optimize quality in Breast Cancer decision making, management and treatment in the peripheries. Conclusions.: Ergonomics of Innovating Training Networks for Interactive E-Learning in the context of the E-Multidisciplinary Board in cancer decision support and making, management and treatment in combination with the Mobile Clinical Unit ‘’Macedonia’’ for Breast Cancer prevention may be instructed remotely and interactively, while integration with AI and Big Data analytics interactive E-Learning may teach cognitive ergonomics of the E-Multidisciplinary Board and sustain an efficient surveillance system on a European Level and in the Balkans to optimize quality in Breast Cancer Screening, Prevention, Treatment and Survivorship especially for the peripheries.
Blind Decryption and Private Information Delivery

We analyze the privacy protection scheme given by Bao, Deng and Feng which is a cryptographic protocol similar to oblivious transfer and private information retrieval schemes. Their scheme is based on a commutative family of ciphers. We point out several security flaws caused by use of an inadequate commutative family of ciphers. Moreover, we remedy the defects by proposing a new scheme that is regarded as an extension of their scheme. We use an approach different from them; we formalize a double encryption and blind decryption scheme employing the ElGamal encryption algorithm and apply it to realize a private information delivery scheme which is an extension of the privacy protection scheme.
Involvement (With Technology)

We live in a culture which largely defined by our relationship with digital technology. This technology for most people begins (and ends) with their mobile phone. We use digital products to mediate just about every aspect of our lives, and we are intimately involved with it.
A Blockchain-Based Framework for Audio Copyright Deposition

Based on the defects of existing audio copyright transaction processes such as complicated deposition models, long periods of rights maintenance, and ambiguous benefit distribution, this paper mainly proposes a model of depositing audio with unique identification on a real and transparent blockchain. Based on blockchain technology, which is famous for its decentralization, powerful tamper-proof function, and smart contract, the model transforms the audio data uploaded by music owners into uniquely identified audio fingerprints using the audio fingerprint recognition library (Dejavu) in python. Through the signature of the user’s private key and the confirmation of each node in the blockchain, it is finally uploaded and broadcasted to complete a series of tamper-evident deposition processes. In this way, based on the decentralization, traceability, and anti-tampering features of blockchain, the audio owner can transact directly with the buyer, and the process of defending rights is greatly simplified and has strong legal proof.
Multi-view Fuzzy Representation Learning with Rules based Model

Unsupervised multi-view representation learning has been extensively studied for mining multi-view data. However, some critical challenges remain. On the one hand, the existing methods cannot explore multi-view data comprehensively since they usually learn a common representation between views, given that multi-view data contains both the common information between views and the specific information within each view. On the other hand, to mine the nonlinear relationship between data, kernel or neural network methods are commonly used for multi-view representation learning. However, these methods are lacking in interpretability. To this end, this paper proposes a new multi-view fuzzy representation learning method based on the interpretable Takagi-Sugeno-Kang (TSK) fuzzy system (MVRL_FS). The method realizes multi-view representation learning from two aspects. First, multi-view data are transformed into a high-dimensional fuzzy feature space, while the common information between views and specific information of each view are explored simultaneously. Second, a new regularization method based on L2,1-norm regression is proposed to mine the consistency information between views, while the geometric structure of the data is preserved through the Laplacian graph. Finally, extensive experiments on many benchmark multi-view datasets are conducted to validate the superiority of the proposed method.