Reviewerly/exHarmony huggingface dataset:
An Interactive System for Multimedia Retrieval in Video Collection with Temporal Integration

Multimedia retrieval in computer science is the process of obtaining text, images, videos, and audio segments, all in digital form relevant to an information need from a collection of these resources. With the ever-growing amount of data, scalable and interactive retrieval systems that can efficiently work on extensive data collections while maintaining high precision are in high demand by industries and researchers. This paper presents the Pumpkin system, an interactive multimedia retrieval system first used in The AI Challenge Ho Chi Minh City 2023, an annual video event and moment retrieval competition. The system is built and set in motion to handle the retrieval task in a video collection of considerable size and complexity by three primary methods: visual-text association search, object-based search, and audio speech instances search. Additionally, the system has an integrated temporal workflow to search for conceptually related shots in a sequential motion, which removes out-of-context while leveraging suitable results as the user inputs more details to the system. Our system also puts great emphasis on user experience by cooperating with a clean and intuitive interface design with simplified user-side functionality, allowing a more efficient process of information retrieval, whether primary or complex, in a huge collection of multimedia data.
Improving Multilingual Neural Machine Translation with Artificial Labels

Inspired by the work which uses Artificial Translation Units for generation of synthetic data in low-resource Neural Machine Translation systems [12], we propose using these translation units to enhance ability of sharing information between translation units in the multilingual Neural Machine Translation systems. In particular, we concentrate on improving the translation of rare-words. Our method also suggest a new idea about leveraging bilingual dictionaries in multilingual Neural Machine Translation systems which is still limited in prior works. Our experiments show improvements of up to +3.5 BLEU scores in the translation tasks between Chinese, Japanese and Vietnamese from the TED Talks domain. Our machine translation system outperforms the systems in [12] when translating from Chinese to Vietnamese although we do not use any additional techniques such as data argumentation or pre-trained model as shown in [12].
Diverse Search Methods and Multi-Modal Fusion for High-Performance Video Retrieval

Querying events within extensive video datasets currently stands as a prominent research focus within the field of multimedia information retrieval. Achieving high-performance retrieval within such contexts necessitates the efficient extraction and effective storage of information from videos to expedite the retrieval process. These challenges become notably pronounced when handling substantial datasets. In this paper, we introduce a system tailored for event querying within video data. Our system is meticulously crafted to optimize information retrieval speed and to efficiently organize storage, harnessing the power of FAISS and ElasticSearch. It boasts the capability to process diverse forms of input information, including textual video descriptions, Optical Character Recognition (OCR) results, Automatic Speech Recognition (ASR) transcriptions, visually similar images, and details about objects within videos, encompassing aspects such as color and quantity. Moreover, our system can also extract information about the temporal sequence of events within videos—a particularly challenging task when extracting information from video frames. By amalgamating these various input types, our system delivers optimal results.
Lite FPN_SSD: A Reconfiguration SSD with Adapting Feature Pyramid Network Scheme for Small Object Detection

Detecting small objects poses a significant challenge in computer vision because of the low resolution and fuzzy feature representation. Although one-stage detection techniques alleviate the problem caused by scale difference to some extent, they also retain redundant features, resulting in resource wastage and slower processing speeds. This research first primarily centers around elucidating the SSD algorithm, a new improved framework, namely Lite FPN-SSD (Lite Single Shot Multibox Detector with Adapting Feature Pyramid Network), then is proposed to solve the weakness of the SSD algorithm. The Lite FPN-SSD is build upon the popular FPN and SSD architectures to create a learnable fusion scheme with controlling the feature information that deep layers deliver to shallow layers. Its lightweight nature, with a minimal increase in parameters, ensures high efficiency for real-time applications. Extensive experiments conducted on VOC, VEDAI and SOHAS datasets demonstrate an impressive results of the proposed models in comparison with the original SSD and its other variations. Particularly, by making a minimal addition of only 2 million parameters, the proposed model achieves a mean average precision (mAP) of 78.36% on the VOC dataset, which is close to another architecture that achieved a 78.40% mAP but require more than 2.6.
Video Sage: Video Recommendation Using Graph Convolution Neural Network

Video recommendation has become an indispensable means of helping individuals navigate through the vast expanse of videos and discover content that aligns with their interests. In current video recommendation systems, recommendations are made based on user-video interactions and specific individual content features. Despite the richness of information contained within videos, the challenge lies in effectively representing and extracting distinctive features that can be presented to users. Presently, video recommendation models predominantly rely on isolated video features for suggestions, without effectively harnessing the available information by combining these features cohesively. In this article, we introduce a video model called Video Sage, which integrate numerous video characteristics with a user's interaction history. The model is rooted in the concept of Graph Convolution Neural Network (GCN), which can discern user-video interactions and maintain scalability for future data. Additionally, we propose a solution for embedding and merging complex video features, which serve as input to the Video Sage model, enhancing recommendation efficiency. Our method has been rigorously tested on a dataset comprising users' video viewing histories on social networks and has yielded exceptionally competitive results.
Leveraging FFT and Hybrid EfficientNet for Enhanced Action Recognition in Video Sequences

Human Action Recognition (HAR) has emerged as a pivotal challenge in the rapidly evolving realm of artificial intelligence and computer vision. This research delves into enhancing action recognition in video sequences by synergizing the Fast Fourier Transform (FFT) capabilities and the EfficientNet model. We employed the UCF101 dataset, encompassing a diverse range of action categories, to validate our approach empirically. The results underscored that incorporating FFT in the preprocessing stage amplifies the recognition efficacy of the EfficientNet model. Comparative analyses further substantiated the superiority of this combined approach over conventional methods, marking a significant contribution in the domain of action video recognition.
Language Knowledge-Assisted in Topology Construction for Skeleton-Based Action Recognition

Skeleton-based action recognition is a challenging problem due to the high dimensionality and noisy nature of skeleton data. Graph convolution networks (GCNs), which use graph topology to extract representative features, have been effective for skeleton-based action recognition in recent years. However, effectively learning and aggregating topology information is challenging problem. In this work, we propose a strategy to construct topology representation to skeleton-based action recognition that combines language knowledge to learn the topology. Specifically, borrows the idea from Language Knowledge-Assisted Representation Learning (LAGCN) [20], which uses a large-scale language model (LLM) to learn a priori global relationship (GPR) topology that captures the global structural relationships between the joints and a priori category relationship (CPR) topology between nodes in the skeleton graph to capture the category-specific relationships between the joints. We propose to apply the GPR topology as a prior topology, provide significant momentum to learn the model along with the CPR which is used to learn the class-distinguishable features into the Channel-wise Topology Refinement Graph Convolution (CTRGCN) [4]. The proposed approach is evaluated on the NTU RGB+D and NW-UCLA datasets. The results show that the proposed approach achieves promising results with 96.76% on the NW-UCLA dataset and 97% on the NTU dataset with the cross-view benchmark along with 92.8% on NTU cross-subject benchmark.
Multi-User Video Search: Bridging the Gap Between Text and Embedding Queries

Video search is a crucial task in the modern era, as the rapid growth of video platforms has led to an exponential increase in the number of videos on the internet. Effective video management is therefore essential. Significant research has been conducted on video search, with most approaches leveraging image-text retrieval or searching by object, speech, color, and text in images. However, these approaches can be inefficient when multiple users search for the same query simultaneously, as they may overlap in their search spaces. Additionally, most video search systems do not support complex queries that require information from multiple frames in a video. In this paper, we propose a solution to these problems by splitting the search space for different users and ignoring images that have already been considered by other users to avoid redundant searches. To address complex queries, we split the query and apply a technique called forward and backward search.
Applying Adaptive Sharpness-Aware Minimization to Improve Out-of-distribution Generalization

Out-of-distribution (OoD) generalization in machine learning occurs when models trained on specific source domains struggle to generalize to unseen target domains due to variations in distributions, data collection conditions, and biases. These differences, known as domain shifts, can considerably impact model performance when facing domains not encountered during training. This circumstance arises partially due to the limitations of commonly used optimizers like SGD or ADAM, which lack the capability to preferentially converge towards optimal points characterized by high generalization capacity, commonly referred to as flat minima. Sharpness-Aware Minimization (SAM) emerges as a powerful tool for facilitating generalization in independent and identically distributed (i.i.d) data by aiming to find flat minima in the loss landscape that remain robust in the face of input sample perturbations. Among the variants of SAM, one notable variant is ASAM, an adaptive version designed to withstand parameter re-scaling and ensure robustness. In this study, our primary focus lies in the utilization of ASAM. Diverging from the conventional utilization of ASAM, which predominantly functions within the confines of i.i.d conditions, this paper applies ASAM in OoD data scenarios to evaluate its robustness. Our findings reveal that ASAM exhibits stable training behavior and superior generalization capabilities when compared to the Adam optimizer. Our experiments, conducted on the NICO dataset encompassing multiple domains, showcase ASAM's remarkable performance, surpassing other state-of-the-art methods without the need for intricate data augmentation. ASAM achieves an accuracy rate of 87.25% (Animal) and 80.79% (Vehicle).
Optimizing Results in Aerial Images through Post-Processing Techniques on YOLOv7

Object detection in aerial images has garnered significant attention from the research community in recent years. The challenges posed by small objects, diverse orientations, and complex backgrounds have spurred extensive research efforts. In this paper, we focus on object detection in a YOLOv7-based framework, to address the issue of overlapping predicted regions, we introduce and apply the Soft-NMS (Non-Maximum Suppression) technique, a post-processing method known for its effectiveness in improving detection accuracy in such scenarios. Soft-NMS adjusts bounding box scores based on the extent of overlap, allowing for more accurate localization of objects in densely populated regions. Furthermore, we present comprehensive experimental results to validate the efficacy of our proposed approach. The analyses encompass a thorough evaluation on the UCAS Aerial Object Detection (UCAS AOD) dataset, comprising over 1500 aerial images captured from diverse perspectives. Our method has demonstrated an improvement in object detection performance, particularly in scenarios with closely positioned objects. The proposed framework showcases its ability to handle complex aerial scenes with higher precision and recall rates compared to conventional methods.
A Generalized Autorec Framework Applying Content-based Information for Resolving Data Sparsity Problem

Recent years have witnessed several neural network-based collaborative filtering systems that yield immense success in providing users with personalized suggestions. These systems, however, severely suffer from the data sparsity problem. Similar to other neural network-based collaborative filtering systems, the sparse rating information also affects Autorec at the input layer. This paper represents a generalized hybrid Autorec framework that can accept a variety of content-based representations as input to create a more efficient system. Subsequently, to fully employ the capability of the Autoencoder architecture in compressing information, rating vectors compressed via the conventional Autorec, together with content-based information, creates a robust hybrid input for our framework to resolve the sparsity problem. Empirical experiments demonstrate the state-of-the-art performance of our system compared to other hybrid recommendation systems.
OsteoGA: An Explainable AI Framework for Knee Osteoarthritis Severity Assessment

Knee osteoarthritis is among the most common joint disorders. Recent studies have investigated the application of Artificial Intelligence (AI) in automated diagnosis using knee joint X-ray images. However, these studies have primarily focused on diagnosing the severity of osteoarthritis without providing explanations for the underlying reasons that led to those results. In this paper, we present OsteoGA, an AI framework that focuses on the interpretability of the model in order to assist in the diagnosis of knee osteoarthritis. OsteoGA introduces a novel generative adversarial autoencoder model, called GAE, to reconstruct an assumed healthy knee joint image from the original ones. The reconstruction process in OsteoGA combines image reconstruction, data imputation, and adversarial learning to generate high-quality images that are consistent and coherent with the patient's image. Apart from effectively diagnosing the severity of knee osteoarthritis, the OsteoGA framework also produces an anomaly map. This map highlights valuable information about the abnormal regions in X-ray images, offering additional insights to medical experts during the diagnosis process.
An Ensemble Approach to Graph Layout for Movie Poster Similarity Visualization

An integral element within the realm of movie-making is the movie poster image. This visual medium holds significant importance, conveying essential details about the movie, including its title, characters, and genre. Hence, the features extracted from the movie poster are valuable for measuring the similarity of the movies. This study proposes an ensemble approach to identify movie similarity for visualizing the similarity of movie posters. Specifically, we consider three methods i) reference-based, ii) traditional feature-based, and iii) deep learning feature-based regarding constructing the similarity matrix. This way, we can estimate the movie similarity more accurately to provide it as the input dataset for personalized movie recommendation systems. In particular, our method focuses on extending the color-based search algorithms that use low-level image histogram features. This approach helps the classification processing time increase, leading to the quick generation and comparison of feature vectors.
A Systematic Literature Review of DevOps Success Factors and Adoption Models

This paper investigates DevOps adoption, emphasizing its cultural dimensions in the context of the information systems domain. Utilizing a robust systematic literature review methodology, we discerned nine salient success factors and delineated three stage models instrumental for a smooth transition into DevOps-centric projects. Despite the increasing breadth of research in the DevOps area, the absence of a consolidated definition remains a challenge, underscoring the importance of a unified interpretative framework. This research notably extends the current body of knowledge by focusing on the cultural dynamics of DevOps adoption. As a result, it offers a refined lens through which both practitioners involved in the adoption process and academic researchers can comprehend the nuances and implications of cultural shifts in the evolving landscape of DevOps within information systems.
A Comprehensive Web Annotation Application for Organ Image Segmentation and Predictive Inference

In this paper, we present a web annotation application tailored to streamline the process of annotating 3D organ volumes using preprocessed 2D images data. Our platform introduces advanced annotation assistance and predictive inference to significantly enhance medical imaging workflows. Although we leverage curated sets of 2D images rather than the original 3D volume data, our innovative approach focuses on optimizing the annotation process and integrating predictive algorithms to improve accuracy. By capitalizing on the preprocessing step of 3D volume segmentation dataset, we offer a solution that expedites the creation of accurate organ segmentation, ultimately contributes to more efficient medical image analysis.
A New Transfer Learning-Based Traffic Classification Algorithm for a Multi-Domain SDN Network

To enhance the efficiency and resource utilization of a computer network, it is imperative to classify network traffic and implement distinct priority policies. Network traffic classification plays a pivotal role across various domains, including network administration, cybersecurity, and network resource optimization. As encrypted network data undergoes diverse evolution, as evident in datasets from tech giants like Google, Facebook, and YouTube, traditional traffic classification methods have given way to machine learning-based approaches. Given that computer networks are primarily deployed as distributed multi-domain systems, employing machine learning for traffic classification becomes challenging when a new network domain appears with a limited dataset. One potential remedy is to employ transfer learning, allowing knowledge transfer from a pre-trained model in an established domain to a new one. In this paper, we present two contributions. First, a novel algorithm called Multi-class TrAdaBoost-CNN is introduced to tackle the challenge of cross-domain classification in encrypted network services. This algorithm extends the Multi-class TrAdaBoost approach by incorporating a Convolutional Neural Network (CNN) as a weak learner. Secondly, extensive experiments are conducted on two distinct domains characterized by imbalanced data distributions to assess the efficacy of our proposed method. The experimental results clearly demonstrate that our algorithm outperforms the traditional CNN model, achieving remarkable accuracy improvements of up to 16%, even when dealing with extremely limited data.
On Remoteness Functions of Exact Slow NIM, NIM$^1_=(4,2) and NIM^1_=(5,2)

In three recent works: https://arxiv.org/abs/2304.06498,https://arxiv.org/abs/2311.13511, and https://arxiv.org/abs/2311.03257 a very simple explicit rule was suggested and applied to compute Smith's remoteness function of the game Exact Slow NIM$_k$ with $n$ piles of stones for the case $n=k+1$. In this paper we extend this rule to the general case of Exact Slow NIM, NIM$^1_=(n,k)$, and apply it for $n=4, k=2$ and $n=5, k=2$. In both cases it works efficiently and reduces the remoteness function by 1 with each move in ``almost all" cases. The exceptions are sparse and have regular pattern. We suggest simple formulas that cover all found exceptions.
GPU-Based Parallel Path Planning for Mobile Robot Navigation in Dynamic Environments

In our modern, ever-evolving, and intricately interconnected world, the task of planning optimal paths for mobile robots has emerged as a challenge. This challenge arises from our surroundings' growing dynamism and complexity, characterized by the continual variation of obstacles and environmental conditions. This paper proposes a new approach called the Parallel Kinematic Rapidly Exploring Random Tree (PK-RRT) algorithm for mobile robot path planning in dynamic environments. Our PK-RRT algorithm incorporates kinematic constraints when the robot moves on the RRT tree to ensure that the paths generated are feasible and smooth. It also applies the multi-threading technique in which multiple threads work in parallel to speed up the path generation process and improve its convergence. We conducted a comprehensive performance comparison with two existing algorithms, namely Bidirectional Rapidly Exploring Random Tree (Bi-RRT) and Quadratic Rapidly Exploring Random Tree (Quad-RRT). The results show that our PK-RRT algorithm reduces the computation time by 30%, improves the computational efficiency by 40%, and increases the stability by 50% compared to the existing algorithms.
Automated generation of adaptive perturbed images based on GAN for motivated adversaries on deep learning models: Automated generation of adaptive perturbed images based on GAN

Deep learning techniques have achieved great success in many fields, such as computer vision, natural language processing and computer security. However, deep learning models are facing many security risks, in particular motivated adversaries which lead to incorrect predictions or reduce the models' effectiveness. Many previous researches have been investigated on adversarial attacks, aiming to improve the robustness and security of deep learning models. In this article, we propose a method to automatically produce adaptive perturbed images based on GAN for motivated adversaries on deep learning models. The experimental results achieve approximately 60% success rate in evading five state-of-the-art deep learning models for image recognition including ResNet-56, MobileNetV2, VGG19_bn, ShuffleNetV2, RepVGG_a2 on the CIFAR-10 dataset. The proposed method's results are much higher than AIGAN model proposed by Tao Bai et al. achieving 10.17%. The image fidelity of distorted images generated by the proposed method also is positive. The proposed method's PSNR (Peak-Signal-Noise-Ratio) is greater than 40 compared to previous studies such as FGSM, DeepFoll, C&W and AdvGAN, Zhang in which PSNR is less than 30.