Reviewerly/exHarmony huggingface dataset:
I2SRM: Intra- and Inter-Sample Relationship Modeling for Multimodal Information Extraction

Multimodal information extraction is attracting research attention nowadays, which requires aggregating representations from different modalities. In this paper, we present the Intra- and Inter-Sample Relationship Modeling (I2SRM) method for this task, which contains two modules. Firstly, the intra-sample relationship modeling module operates on a single sample and aims to learn effective representations. Embeddings from textual and visual modalities are shifted to bridge the modality gap caused by distinct pre-trained language and image models. Secondly, the inter-sample relationship modeling module considers relationships among multiple samples and focuses on capturing the interactions. An AttnMixup strategy is proposed, which not only enables collaboration among samples but also augments data to improve generalization. We conduct extensive experiments on the multimodal named entity recognition datasets Twitter-2015 and Twitter-2017, and the multimodal relation extraction dataset MNRE. Our proposed method I2SRM achieves competitive results, 77.12% F1-score on Twitter-2015, 88.40% F1-score on Twitter-2017, and 84.12% F1-score on MNRE. 1
Feature Enhancement and Foreground-Background Separation for Weakly Supervised Temporal Action Localization

Weakly-supervised Temporal Action Localization (W-TAL) is a significant task in video understanding, intending to recognize the category and pinpoint the temporal boundaries of action segments in untrimmed videos based on the video-level labels. Due to the lack of frame-level annotations, recognizing the spatiotemporal relationships among action snippets and precisely separating the foreground and background are two arduous challenges. In this paper, we propose a novel Feature Enhancement and Foreground-Background Separation (FE-FBS) method to address these problems. Specifically, we construct a spatiotemporal cooperation enhancement scheme utilizing residual graph convolutional to capture the spatial and temporal dependencies between the current snippet and others, generating snippet features with spatial and temporal correlations, thereby ensuring a complete feature representation for action localization. Furthermore, we propose a two-branch explicit foreground-background joint attention mechanism to guide foreground and background modeling, combined with an inverse enhancement strategy to enhance action weight for better foreground and background distinction, thereby improving the accuracy of action localization. We use THUMOS’14 and ActivityNet v1.3 datasets to achieve accuracy rates of 66.7% and 38.9%, respectively, and compare our method to other methods, gaining superiority.
FTUnet: Feature Transferred U-Net For Single HDR Image Reconstruction

The development of the display technology supports the application of High Dynamic Range (HDR) enabling devices. In order to meet the surging demand for the HDR media content, we propose a feature-transferred U-shaped network (FTUnet) to convert existing Standard Dynamic Range (SDR) images into their HDR counterparts. The proposed FTUnet is a feature transformation network that converts the encoded SDR features to the HDR features. This transformation network extracts features rich of spatial information by a self-attention mechanism, in order to improve the reconstruction of the over-exposed regions and avoid unreasonable patches. Besides, we propose an Excitation-Restoration (ER) sub-network to involve the inter-channel attention mechanism. The ER network is used to remove redundant information between channels and reserve the key features. Therefore, the proposed FTUnet can efficiently merge feature channels and contribute to the advantage in color accuracy for the generated HDR images. Experimental results show that our proposed FTUnet achieves state-of-the-art performance in both quantitative comparison and visual quality for the single HDR image reconstruction. The ablation study is also performed to demonstrate the effectiveness of each module of the proposed FTUnet.
ADNet: An Asymmetric Dual-Stream Network for RGB-T Salient Object Detection

RGB-Thermal salient object detection (RGB-T SOD) aims to locate salient objects in images that include both RGB and thermal information. Previous approaches often suggest designing a symmetric network structure to tackle the challenge of dealing with low-quality RGB or thermal images. However, we contend that RGB and thermal modalities possess different numbers of channels and disparities in information density. In this paper, we propose a novel asymmetric dual-stream network (ADNet). Specifically, we leverage an asymmetric backbone to extract four stages of RGB features and four stages of thermal features. To enable effective interaction among low-level features in the first two stages, we introduce the Channel-Spatial Interaction (CSI) module. In the last two stages, deep features are enhanced using the Self-Attention Enhancement (SAE) module. Experimental results on the VT5000, VT1000, and VT821 datasets attest to the superior performance of our proposed ADNet compared to state-of-the-art methods.
Geometric Style Transfer for Face Portraits

Geometric style transfer jointly stylizes the texture and geometry of a content image to better match a style image, which has attracted widespread attention due to its various applications. However, existing style transfer methods either primarily focus on texture and almost entirely ignore geometry, or have various drawbacks and are not suitable for Face Portraits. In the paper, We propose a new two-stage geometric style transfer method dedicated to face portraits, which simultaneously transfer both statistical and structural styles. Our network consists of Geometric deformation module (G) and Texture rendering module (T). G is trained with semantics image pairs, which has loose requirements on the training datasets. Besides, our flexible formulation also allows explicit user guidance and control of stylization tradeoffs. Experiments demonstrate that our method achieves state-of-the-art geometric style transfer for face portraits.
Generic Attention-model Explainability by Weighted Relevance Accumulation

Attention-based Transformer models have achieved remarkable progress in multi-modal tasks, such as visual question answering. The explainability of attention-based methods has recently attracted wide interest as it can explain the inner changes of attention tokens by accumulating relevancy across attention layers. Current methods simply update relevancy by equally accumulating the token relevancy before and after the attention processes. However, the importance of token values is usually different during relevance accumulation.In this paper, we propose a weighted relevancy strategy, which takes the importance of token values into consideration, to reduce distortion when equally accumulating relevance. To evaluate our method, we propose a unified CLIP-based two-stage model, named CLIPmapper, to process Vision-and-Language tasks through CLIP encoder and a following mapper. CLIPmapper consists of self-attention, cross-attention, single-modality, and cross-modality attention, thus it is more suitable for evaluating our generic explainability method. Extensive perturbation tests on visual question answering and image captioning tasks validate that our explainability method outperforms existing methods.
Graph-Guided MLP-Mixer for Skeleton-Based Human Motion Prediction

In recent years, Graph Convolutional Networks (GCNs) have been widely used in human motion prediction, but their performance remains unsatisfactory. Recently, MLP-Mixer, initially developed for vision tasks, has been leveraged into human motion prediction as a promising alternative to GCNs, which achieves both better performance and better efficiency than GCNs.
MA-Net: Multi-Attention Network for Skeleton-Based Action Recognition

Graph Convolution Networks (GCNs) have become the main-stream framework for skeleton-based action recognition tasks. Aiming at the problem of redundant spatial-temporal feature information and neighborhood constraints obtained in GCNs, we propose a novel method called Multi-Attention Network (MA-Net) to explore crucial skeleton information, including two main modules: Combined Attention Graph Convolution (CAGC) and Multi-layer Transposed Attention Encoding (MTAE). The CAGC utilizes multi-dimensional combination attention to capture more valuable information and enhance feature performance. The MTAE adopts self-attention to encode feature maps, effectively establishing long-range dependency and capturing global information. Centre on the attention mechanism, these two modules combine the complementary advantages of GCN (i.e., local topology and temporal dynamics) and Transformer (i.e., global context and dynamic attention). Extensive experiments on the challenging NTU-RGB+D 60 and Kinetics-Skeleton datasets demonstrate that our model performs excellently.
Key Parts Spatio-Temporal Learning for Video Person Re-identification

Person re-identification (Re-ID) is a technology to identify specific pedestrians in different scenarios. In recent years, Re-ID has been widely used in surveillance, supermarket and smart city. However, there are still many challenges in this field, including complex background, pose changes, and occlusion dislocation. We propose a novel Key Parts Spatio-temporal Learning (KSTL) framework to alleviate the above problems. Specifically, we first use the mask method based on keypoint detection to locate and extract the key part features of the human body. Then, we introduce Spatio-temporal Learning (STL) block based on key parts to realize the mutual transfer and learning of key parts features of multiple frames. Finally, we fuse the learned key part features and global features as the final video representation. The method we propose can not only accurately learn the features of key parts, but also make full use of the timing information in the video, thus achieving good detection results. We conduct extensive experiments on three public benchmarks, and the results demonstrate the effectiveness and superiority of KSTL.
Relevance and Irrelevance Considered Subspace Mapping Neural Networks for Remote Sensing Text-Image Retrieval

Remote sensing cross-modal image-text retrieval has attracted increasing attention due to its important roles in multiple domains. Existing methods perform salient modeling for the feature that has high relevance between different modalities. However, most works consider the relevance between different modalities but ignore the irrelevance between different modalities, resulting in incomplete modeling of the relevance and irrelevance between different modalities. In this paper, we propose a Relevance and Irrelevance Considered Subspace Mapping Neural Networks (RIR-SMNNs) to simultaneously consider the relevance and irrelevance between different modalities. Specifically, we first utilize Multiscale Image Feature Extraction (MIFE) and Multiscale Text Feature Extraction (MTFE) to extract the multiscale feature of image and text. Then, we perform Local Space Building Module (LSB), which constructs local space that realizes scale alignment. Finally, we perform the Relevance and Irrelevance Local Space Mapping (RIRLSM) to consider the relevance and irrelevance of different modalities in multiple spaces. Experimental results on several remote sensing datasets demonstrate our model outperforms the state-of-the-art approaches.
Hierarchical Multi-Scale Adaptive Conv-LSTM Network for Human Action Recognition Based on Wearable Sensors

Recently, human action recognition has been widely used in the fields of health monitoring, human-robot interaction, medical treatment, and sports. Due to the availability of various wearable devices on the market, we can easily access sensor data for human action recognition. However, it is still a challenge to capture minute action processes as well as extract spatio-temporal motion patterns from serial sensor data. Therefore, we propose a novel hierarchical multi-scale adaptive Conv-LSTM network structure called HMA Conv-LSTM. The finer-grained spatial information in the sensor signals is extracted by hierarchical multi-scale convolution. The multi-channel feature fusion through adaptive channel feature fusion retains important information and improves model efficiency. We capture temporal context information by dynamic channel selection-LSTM based on the attention mechanism. Extensive experiments on the Opportunity and PAMAP2 public datasets show that our proposed model achieves competitive performance compared to several state-of-the-art approaches.
Multi-view–enhanced modal fusion hashing for Unsupervised cross-modal retrieval

Cross-modal hashing is an important direction for multimodal data management and applications, which has recently received more and more attention. Unsupervised cross-modal retrieval does not rely on tag information and is more applicable to the real world. However it still faces some problems. Existing methods mainly encode for local features or global features. Due to the effect of negative samples, it is easy to cause noise interference. To solve these problems, we propose a Multi-view–enhanced modal fusion hashing for Unsupervised cross-modal retrieval (MUCH) to improve these problems. Firstly, we propose a multi-view network. Images inherently contain richer semantics, and we employ a multi-view network to observe the image from different perspectives and obtain the overall and local features of the image. Secondly, we introduce a noise cancellation module to approximate the cross-modal data feature alignment from both intra-modal and cross-modal perspectives before generating the hash code. Finally, we construct a distribution-based similarity weighting matrix to replace the graphical similarity matrix. And we performed multi-view enhancement experiments on JDSH and CIRH, with 1% to 2% enhancement over DAEH on all three datasets.
AniCropify: Image Matting for Anime-Style Illustration

Recently, deep learning-based image matting methods have emerged. However, the existing methods lack the capability to provide precise matting for anime-style illustrations because their network parameters are trained on primarily photo-realistic images. In this paper, we introduces a new anime image dataset, Chara-1M, designed for matting purposes. In addition, we propose AniCropify, a new matting method for character anime images. Focusing on the commonalities of representation between anime images and photo-realistic images, in AniCropify, an anime image is first converted into a photo-realistic image. From the converted image, a trimap is generated to identify the human regions in images. By using the trimap in the matting process, precise alpha masks of anime images can be obtained. From experiments, we confirmed that based on the quality evaluation of matting results, the proposed method received the highest rating compared to other state-of-the-art techniques.
Optical Flow based Feature Prediction and Decomposed Context for Video Compression

In recent years, there have been a growing interest in developing end-to-end neural video codecs. Previous works generally use a past decoded frame as reference directly, utilizing the motion information between it and the input frame to reduce temporal redundancy. However, this approach may lead to high bit rate consumption of the motion and fails to take advantage of the prior information in other reconstructed frames. In this work, We propose a learned video coding framework with optical flow based feature prediction module and decomposed context module. Specifically, we employ the previous optical flow to generate a warped frame, and along with other reconstructions, they are used for a more accurate reference forecasting, thereby reducing the bit rate required for motion compression. Moreover, based on the conditional coding framework, our decomposed context module explores conditional context in past decoded frames and further reduces additional spatiotemporal correlations. Experimental results demonstrate that our approach yields better performance than previous learned video compression methods and traditional standard codecs. For example, our neural codec achieves 28.94% coding gain over HEVC in PSNR metric and about 2.00% coding gain over VVC in MS-SSIM metric.
Using Carbon Sequestration as a Remote-Monitoring Approach for Reclamation’s Effectiveness in the Open Pit Coal Mine: A Case Study of Mae Moh, Thailand

Reclamation is regarded as one of the mining processes that can lessen the environmental impact of its production, particularly for large-scale coal mines that emit significant quantities of greenhouse gases. However, the assessment and evaluation of the reclamation process primarily rely on qualitative methods. Utilizing LANSAT8 Operational Land Imager (OLI) remote sensing in conjunction with GIS, this study aimed to develop a quantitative method for validating the efficacy of a reclamation procedure applicable to the emerging trend of carbon reduction. The empirical formula utilized to compute the annual carbon sequestrations of the reclamation area in the Mae Moh mine exhibited the appropriate spatial relative standard deviation (S-RSD) at 98.25%. The findings indicate that the reclamation area reached its highest level of carbon sequestration in 2022, at 331.28 ± 11.89 ktCO2e, surpassing the baseline of 126.53 ktCO2e. Furthermore, the approach demonstrates significant potential in improving the standard method for assessing reclamation through reforestation.
Elementary Number Theory Problems. Part XI

Summary In this paper we present the Mizar formalization of the 36th problem from W. Sierpiński’s book “250 Problems in Elementary Number Theory” [10].
Multidimensional Measure Space and Integration

Summary This paper introduces multidimensional measure spaces and the integration of functions on these spaces in Mizar. Integrals on the multidimensional Cartesian product measure space are defined and appropriate formal apparatus to deal with this notion is provided as well.
The Ring of Conway Numbers in Mizar

Summary Conway’s introduction to algebraic operations on surreal numbers with a rather simple definition. However, he combines recursion with Conway’s induction on surreal numbers, more formally he combines transfinite induction-recursion with the properties of proper classes, which is diffcult to introduce formally. This article represents a further step in our ongoing e orts to investigate the possibilities offered by Mizar with Tarski-Grothendieck set theory [4] to introduce the algebraic structure of Conway numbers and to prove their ring character.
Elementary Number Theory Problems. Part X – Diophantine Equations

Summary This paper continues the formalization of problems defined in the book “250 Problems in Elementary Number Theory” by Wacław Sierpiński.
Increasing the Students’ Speaking Ability through Role-Playing with Slovin’s Formula Sample Size

The ability to communicate is required at this time because it allows for effective communication of ideas, information, and other concepts. To enhance speaking skills, one must use the appropriate media, including flashcard learning media employing the role-playing method. The purpose of this study is to determine the extent to which the Role-Playing Card method with Slovin’s Formula as the sample size can increase students' speaking abilities. The purpose of this experimental study is to determine whether the use of Role-Playing Cards can assist students in developing their speaking abilities. This investigation employed a pre-experimental design. This study involved 42 eighth-grade students from SMPN 19 Kota Tangerang Selatan. Information gathered from the pre- and post-tests The results of the analysis indicate that the t-test value (12,195) is significantly greater than the t-table (23,190). This shows that the null hypothesis (H0) or negative hypothesis, namely that role-playing card methods cannot increase students' speaking ability, cannot be accepted, and the alternative hypothesis (H1) or positive hypothesis, namely that role-playing card methods can increase students' speaking ability, must be accepted. This research concludes, based on the results of t-test calculations, that the role-playing card method is an effective medium for helping students hone their speaking skills
Hubungan antara Internet Self-Efficacy dengan Prestasi Belajar Siswa Sekolah Menengah Atas (Fokus: Analisis Cluster dengan Metode K-Means dan Louvain)

Internet self-efficacy (ISE) was initiated by Albert Bandura, self-efficacy (SE) is a widely accepted concept that refers to an individual's belief in his ability to complete certain behaviors or tasks in certain situations. The purpose of this study was to determine the relationship between ISE and the learning achievement of 12th-grade high school students. The sample of this study was 12th-grade MIPA 7 SMA Unggulan CT ARSA Foundation Sukoharjo as many as 21 students. ISE data is data obtained from questionnaires distributed to students who have been tested for validity and reliability. On the other hand, learning achievement tests were conducted on students with three-dimensional material. The analysis conducted to determine the relationship between the two variables is bivariate analysis. The results showed that between the ISE variable and Student Learning Achievement, a correlation coefficient value of 0.670 was obtained with a significant relationship between the ISE variable and Student Achievement. The degree of bond correlation value shows a strong/sturdy correlation.
Optimizing Curvature and Clearance of Piecewise Bézier Paths

Path optimization typically involves finding a smooth and short path that satisfies motion constraints of a given vehicle or simulated agent. In this paper, we propose a new path optimization scheme based on convex optimization of a piecewise quadratic Bézier path. Our method addresses <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$C^{1}$</tex> continuity and a target minimum clearance to obstacles while minimizing path length and maximum curvature. The user is able to customize the objective function by assigning different weights to the clearance, length, and curvature terms. As a result our method is versatile and suitable for a variety of path planning applications in robotics and autonomous vehicles. We present several results demonstrating the ability to achieve paths according to different values for the curvature, length, and clearance parameters. Our benchmark scenarios apply our method directly to low-quality paths generated by a sampling-based planner, and as well in comparison to a heuristic shortcut-based smoothing method. Our results show that the quality of the paths produced by our method outperforms other approaches, and that our method is responsive to the weights chosen by the user. Our method is able to control the main path properties of interest in a unified fashion, and therefore represents an excellent option for several path planning applications in robotics and autonomous vehicles.