Reviewerly/exHarmony huggingface dataset:
Data-Efficient Masked Video Modeling for Self-supervised Action Recognition

Recently, self-supervised video representation learning based on Masked Video Modeling (MVM) has demonstrated promising results for action recognition. However, existing methods face two significant challenges: (1) video actions involve a crucial temporal dimension, yet current masking strategies adopt inefficient random approaches that undermine low-density dynamic motion clues in videos; (2) pre-training requires large-scale datasets and significant computing resources (including large batch sizes and enormous iterations). To address these issues, we propose a novel method named Data-Efficient Masked Video Modeling (DEMVM) for self-supervised action recognition. Specifically, a novel masking strategy named Flow-Guided Dense Masking (FGDM) is proposed to facilitate efficient learning by focusing more on the action-related temporal clues, which applies dense masking to dynamic regions based on optical flow priors, while sparse masking to background regions. Furthermore, DEMVM introduces a 3D video tokenizer to enhance the modeling of temporal clues. Finally, Progressive Masking Ratio (PMR) and 2D initialization strategies are presented to enable the model to adapt to the characteristics of the MVM paradigm during different training stages. Extensive experiments on multiple benchmarks, UCF101, HMDB51, and Mimetics, demonstrate that our method achieves state-of-the-art performance in the downstream action recognition task with both efficient data and low computational cost. More interestingly, the few-shot experiment on the Mimetics dataset shows that DEMVM can accurately recognize actions even in the presence of context bias.
HSNF: Hybrid sampling with two-step noise filtering for imbalanced data classification

Imbalanced data classification has received much attention in machine learning, and many oversampling methods exist to solve this problem. However, these methods may suffer from insufficient noise filtering, overlap between synthetic and original samples, etc., resulting in degradation of classification performance. To this end, we propose a hybrid sampling with two-step noise filtering (HSNF) method in this paper, which consists of three modules. In the first module, HSNF denoises twice according to different noise discrimination mechanisms. Note that denoising mechanism is essentially based on the Euclidean distance between samples. Then in the second module, the minority class samples are divided into two categories, boundary samples and safe samples, respectively, and a portion of the boundary majority class samples are removed. In the third module, different oversampling methods are used to synthesize instances for boundary minority class samples and safe minority class samples. Experimental results on synthetic data and benchmark datasets demonstrate the effectiveness of HSNF in comparison with several popular methods. The code of HSNF will be released.
ScribbleVC: Scribble-supervised Medical Image Segmentation with Vision-Class Embedding

Medical image segmentation plays a critical role in clinical decision-making, treatment planning, and disease monitoring. However, accurate segmentation of medical images is challenging due to several factors, such as the lack of high-quality annotation, imaging noise, and anatomical differences across patients. In addition, there is still a considerable gap in performance between the existing label-efficient methods and fully-supervised methods. To address the above challenges, we propose ScribbleVC, a novel framework for scribble-supervised medical image segmentation that leverages vision and class embeddings via the multimodal information enhancement mechanism. In addition, ScribbleVC uniformly utilizes the CNN features and Transformer features to achieve better visual feature extraction. The proposed method combines a scribble-based approach with a segmentation network and a class-embedding module to produce accurate segmentation masks. We evaluate ScribbleVC on three benchmark datasets and compare it with state-of-the-art methods. The experimental results demonstrate that our method outperforms existing approaches in terms of accuracy, robustness, and efficiency. The datasets and code are released on GitHub.
Foley gesture: Towards a theory of acousmatic foley

The research project "Acousmatic Foley" addresses common traits between foley art and Concrete Music, based on the idea that the foley artist is an acousmatic listener and, in turn, that acousmatic listening is a form of fiction. In this line, the study argues that both fields have similar treatment of the "sonorous object". For this purpose, the research builds on two lines of thought: the "son-en-scène" and the "mise-en-son". Firstly, the "son-en-scène" focuses on the sounds of the filmic miseen-scène (and its sound props), from very early cases to contemporary instances. The focus on these sound-props provides a perspective of sound for film that emphasizes its role as a tool of fiction and, thus, foley as the craft that leads to that experience. Secondly, "mise-en-son" sheds light on the making of the sound itself by exploring the concept of musical gesture. Either in contexts in which the musical gesture is visible (as with instruments), more cryptic (as with electronic devices), or completely delegated (as in acousmatic music), gesture can be seen a form of agency. Given that foley consists of maneuvering a sound-prop, gesture is as central to foley as it is to musical practices. This paper focus on the idea that gesture carries the same conception as the "sonorous object", that of an "intentional unit". In line with this, and in particular when of acousmatic nature, the research argues that the sonorous object is analogous to the sound-prop. In the end, these two lines of thought (son-en-scènce and mise-en-son) bridge the poietic and esthetic, as in Nattiez's semiotic distinction, towards an experience of "acousmatic foley".
Migration From Relational Database to Column-Oriented NoSQL Database

Relational databases that experience performance issues with processing large amounts of data can benefit from column-oriented NoSQL databases. It offers the advantages of scalability, performance and speed of analytical queries. The migration from relational to column-oriented NoSQL databases is critical because applications that use relational databases are already running, hence it is not possible to use a new NoSQL database without existing data in the relational database. This study introduces two solutions: direct and indirect methods involving schemas and data migration stages. The direct method maps tables to groups of columns, while the indirect method uses ERD extraction and conversion to a NoSQL logical schema. The indirect method outperforms the direct method in many aspects, and the implementation successfully migrated MYSQL databases to Cassandra while maintaining data integrity.
Mixup-Augmented Temporally Debiased Video Grounding with Content-Location Disentanglement

Video Grounding (VG), has drawn widespread attention over the past few years, and numerous studies have been devoted to improving performance on various VG benchmarks. Nevertheless, the label annotation procedures in VG produce imbalanced query-moment-label distributions in the datasets, which severely deteriorate the learning model's capability of truly understanding the video contents. Existing works on debiased VG either focus on adjusting the learning model or conducting video-level augmentation, failing to handle the temporal bias issue caused by imbalanced query-moment-label distributions. In this paper, we propose a Disentangled Feature Mixup (DFM) framework for debiased VG, which is capable of performing unbiased grounding to tackle the temporal bias issue. Specifically, a feature-mixup augmentation strategy is designed to generate new (text, location) pairs with diverse temporal distributions via jointly augmenting the representation of text queries and the location labels. This strategy encourages making prediction based on more diverse data samples with balanced query-moment-label distributions. Furthermore, we also design a content-location disentanglement module to disentangle the representations of the temporal information and content information in videos, which is able to remove the spurious effect of temporal biases on video representation. Given that our proposed DFM framework conducts feature-level augmentation and disentanglement, it is model-agnostic and can be applied to most baselines simply yet effectively. Extensive experiments show that our proposed DFM framework is able to significantly outperform baseline models in various metrics under both independent identical distribution (i.i.d.) and out-of-distribution (o.o.d.) scenes, especially in scenarios with annotation distribution changes.
A Static Detection Method for SQL Injection Vulnerability Based on Program Transformation

Static analysis is popular for detecting SQL injection vulnerabilities. However, due to the lack of accurate modeling of object-oriented database extensions, current methods fail to accurately detect SQL injection vulnerabilities in applications that use object-oriented database extensions. We propose a program transformation-based SQL injection vulnerability detection method to address this issue. This method consists of two stages: program transformation and vulnerability detection. In the first stage, object-oriented database extensions are automatically transformed into semantically equivalent procedural database extensions through the identification of key statements, call relation verification, and program transformation. In the second stage, application programs are automatically scanned using a combination of control flow graph construction and taint analysis techniques to detect SQL injection vulnerabilities. Based on the proposed method, we have implemented the OODBE-SCAN prototype system and performed experimental analysis on eight modern PHP applications. We compare OODBE-SCAN with two related static analysis tools, RIPS and Seay. The results show that OODBE-SCAN can detect more real-world vulnerabilities and has higher accuracy than existing methods.
DPNET: Dynamic Poly-attention Network for Trustworthy Multi-modal Classification

With advances in sensing technology, multi-modal data collected from different sources are increasingly available. Multi-modal classification aims to integrate complementary information from multi-modal data to improve model classification performance. However, existing multi-modal classification methods are basically weak in integrating global structural information and providing trustworthy multi-modal fusion, especially in safety-sensitive practical applications (e.g., medical diagnosis). In this paper, we propose a novel Dynamic Poly-attention Network (DPNET) for trustworthy multi-modal classification. Specifically, DPNET has four merits: (i) To capture the intrinsic modality-specific structural information, we design a structure-aware feature aggregation module to learn the corresponding structure-preserved global compact feature representation. (ii) A transparent fusion strategy based on the modality confidence estimation strategy is induced to track information variation within different modalities for dynamical fusion. (iii) To facilitate more effective and efficient multi-modal fusion, we introduce a cross-modal low-rank fusion module to reduce the complexity of tensor-based fusion and activate the implication of different rank-wise features via a rank attention mechanism. (iv) A label confidence estimation module is devised to drive the network to generate more credible confidence. An intra-class attention loss is introduced to supervise the network training. Extensive experiments on four real-world multi-modal biomedical datasets demonstrate that the proposed method achieves competitive performance compared to other state-of-the-art ones.
BLOCK-FEMF: Efficient Forensic Evidence Management Framework Using Blockchain Technology

Blockchain technology is a revolutionary, secure, and transparent decentralized digital ledger for recording and storing transactions across multiple computers. It is used to secure, track, and manage digital assets, protecting the data from tampering, revision, and deletion. It provides a secure, immutable, tamper-proof ledger to store digital evidence. Blockchain technology can also track the provenance of digital evidence, ensuring that only authenticated and authorized individuals can access the evidence. Existing evidence management systems lack features like authentic intermediate user access and migrating evidence access from one user to another. The existing system uses the Base32 Algorithm, which lacks storage capability, time delay, scalability, and throughput issues. To overcome these issues, in this paper, we propose a Base64 algorithm integrated with IPFS to enhance transaction performance, scalability, and throughput. The Base64 Algorithm encrypts the image evidence and stores it in the blockchain. The IPFS stores images in a decentralized system to optimize the memory and enhance the throughput of transactions in the blockchain environment. The experimental results show that the proposed Base64 Algorithm optimizes memory utilization by 20% and gas utilization by 19.5% and enhances transaction scalability by 20% compared to the existing Base32 Algorithm.
Clip Fusion with Bi-level Optimization for Human Mesh Reconstruction from Monocular Videos

Human mesh reconstruction (HMR) from monocular video is the key step to many mixed reality and robotic applications. Although existing methods show promising results by capturing frames' temporal information, these methods predict human mesh with the design of implicit temporal learning modules in a sequence to frame manner. To mine more temporal information from the video, we present a bi-level clip inference network for HMR, which leverages both local motion and global context explicitly for dense 3D reconstruction. Specifically, we propose a novel bi-level temporal fusion strategy that takes both neighboring and long-range relations into consideration. In addition, different from traditional frame-wise operation, we investigate an alternative perspective by treating video-based HMR as clip-wise inference. We evaluate the proposed method on multiple datasets (3DPW, Human3.6M, and MPI-INF-3DHP) quantitatively and qualitatively, demonstrating a significant improvement over existing methods (in terms of PA-MPJPE, ACC-Error etc). Furthermore, we extend the proposed method on more challenging Multiple Shots HMR task to demonstrate its generalizability. Some visual demos can be seen https://github.com/bicf0/bicf_demo.
Aspect-Based Sentiment Analysis Model with Local Sentiment Aggregation for Online Travel Reviews

Economic growth is a crucial indicator for any country's economy. Tourism plays a vital role in the national economy of Indonesia with its multiplier effect, influencing the development of other sectors. Aspect-based sentiment analysis has emerged as a popular approach for understanding the opinions of tourist reviews. However, most aspect-identification algorithms are based on sentence dependencies, which are computationally expensive for longer texts. Therefore, in this paper, we studied Local Sentiment Aggregation (LSA) as an efficient method to extract aspects and classify sentiment polarities of these aspects. LSA introduces aggregation window-based sentiment learning (AW), a mechanism based on embeddings for neighbouring words. The results of our research demonstrate that the LSA model performs well on tourism reviews, as shown by the model's accuracy, which reached 92.48% and an F1 score of 87.40%.
Intelligent system for text-based diagnostics of deviant states of individual

The article is devoted to the development of the concept of an intelligent system for diagnosing deviant states of individuals based on text analysis using mainly the following text analysis tools: convolutional neural networks, algorithms used by psychiatrists in detecting deviant states of individuals (such as extremism, suicidality, etc.), as well as the theory of information images/representations. Such a system makes it possible to significantly increase the efficiency of identifying such conditions as compared to classical monitoring methods based on content analysis.
Unsupervised Scouting and Layout for Storyboarding in Movie Pre-production

We introduce a system that analyses film content to provide expert camera pose recommendations to cinematographers, so they can rapidly experiment with shots in a virtual space. Shot experimentation to produce storyboards is a crucial preparation step prior to filming, with the time and skill required to create expert storyboards high. Additionally, the convergence between the desired aesthetic and what the physical environment allows is traditionally explored by scouting on location, which can also be costly. We hypothesise that autonomous scouting in the virtual world by moving cameras, actors and lights to achieve specific composition traits is efficient and supports creativity. Our system automates aspects of scouting, storyboarding, and staging without being directed by limiting rule-based cinematography techniques, and instead by data-driven unsupervised clustering.
Gradient-Free Textual Inversion

Recent works on personalized text-to-image generation usually learn to bind a special token with specific subjects or styles of a few given images by tuning its embedding through gradient descent. It is natural to question whether we can optimize the textual inversions by only accessing the process of model inference. As only requiring the forward computation to determine the textual inversion retains the benefits of less GPU memory, simple deployment, and secure access for scalable models. In this paper, we introduce a gradient-free framework to optimize the continuous textual inversion in an iterative evolutionary strategy. Specifically, we first initialize an appropriate token embedding for textual inversion with the consideration of visual and text vocabulary information. Then, we decompose the optimization of evolutionary strategy into dimension reduction of searching space and non-convex gradient-free optimization in subspace, which significantly accelerates the optimization process with negligible performance loss. Experiments in several creative applications demonstrate that the performance of text-to-image model equipped with our proposed gradient-free method is comparable to that of gradient-based counterparts with variant GPU/CPU platforms, flexible employment, as well as computational efficiency.
Promotion Abuse Fraud Detection Application Development using Risk Scoring

Promotion abuse fraud is promotion abuse by duplicating accounts to gain an advantage over promotional codes fraudulently. This action inflicts a financial loss on the company. Therefore, this study aims to deal with fraud by developing an application to detect promotion abuse fraud. The application development process includes needs analysis, modeling, and application development. The dataset used for modeling comes from an e-commerce company in Indonesia. The dataset collection stage includes calculating the similarity between accounts using the Levenshtein distance similarity algorithm to get additional features, such as the number of accounts that are similar to an account. At the modeling stage, experiments were carried out using the Random Forest algorithm and a risk scoring algorithm based on machine learning, namely FasterRisk. The FasterRisk has better performance than the Random Forest algorithm shown by a higher F1 score and AUC score. The algorithm also has an advantage in terms of interpretability because it has an output in the form of a more understandable risk score model, so users can understand the factors that are fraud indicators. The FasterRisk prediction result is then deployed into a web application based on user needs.
Hybrid Interaction Temporal Knowledge Graph Embedding Based on Householder Transformations

Temporal Knowledge Graph Embedding (TKGE) is a crucial technique for performing Temporal Knowledge Graph Completion (TKGC). The effectiveness of TKGE largely depends on the ability to model intrinsic relation patterns. However, as we know, most existing TKGE models usually embed KGs into a single geometric space such as Euclidean, hyperbolic or hyperspherical space to maintain their specific geometric structures (e.g., chain, hierarchy, and ring structures). None of the existing methods can simultaneously model relation patterns of chain, hierarchy, ring structures, and relation mapping properties. This paper constructs a hybrid interaction TKGE model HyIE, which learns spatial structures interactively between the Euclidean, hyperbolic and hyperspherical spaces. HyIE performs two Householder transformations of head and tail entities parameterized by relations in a high-dimensional mixed vector space. The curvature of hyperbolic and hyperspherical spaces depends on the product of both relation and temporal. The core of HyIE lies in implementing transformations and interactions of vectors in Euclidean, hyperbolic and hyperspherical spaces, and Household transformation of head and tail entities. Theoretically, HyIE can model crucial relation patterns and mapping properties simultaneously. Experimental results on five temporal knowledge graph benchmarks show that our HyIE achieves state-of-the-art performance.
Chaos to Order: A Label Propagation Perspective on Source-Free Domain Adaptation

Source-free domain adaptation (SFDA), where only a pre-trained source model is used to adapt to the target distribution, is a more general approach to achieving domain adaptation in the real world. However, it can be challenging to capture the inherent structure of the target features accurately due to the lack of supervised information on the target domain. By analyzing the clustering performance of the target features, we show that they still contain core features related to discriminative attributes but lack the collation of semantic information. Inspired by this insight, we present Chaos to Order (CtO), a novel approach for SFDA that strives to constrain semantic credibility and propagate label information among target subpopulations. CtO divides the target data into inner and outlier samples based on the adaptive threshold of the learning state, customizing the learning strategy to fit the data properties best. Specifically, inner samples are utilized for learning intra-class structure thanks to their relatively well-clustered properties. The low-density outlier samples are regularized by input consistency to achieve high accuracy with respect to the ground truth labels. In CtO, by employing different learning strategies to propagate the labels from the inner local to outlier instances, it clusters the global samples from chaos to order. We further adaptively regulate the neighborhood affinity of the inner samples to constrain the local semantic credibility. In theoretical and empirical analyses, we demonstrate that our algorithm not only propagates from inner to outlier but also prevents local clustering from forming spurious clusters. Empirical evidence demonstrates that CtO outperforms the state of the arts on three public benchmarks: Office-31, Office-Home, and VisDA.