Reviewerly/exHarmony huggingface dataset:
Text-Visual Prompting for Efficient 2D Temporal Video Grounding

In this paper, we study the problem of temporal video grounding (TVG), which aims to predict the starting/ending time points of moments described by a text sentence within a long untrimmed video. Benefiting from fine-grained 3D visual features, the TVG techniques have achieved remarkable progress in recent years. However, the high complexity of 3D convolutional neural networks (CNNs) makes extracting dense 3D visual features time-consuming, which calls for intensive memory and computing resources. Towards efficient TVG, we propose a novel text-visual prompting (TVP) framework, which incorporates optimized perturbation patterns (that we call 'prompts') into both visual inputs and textual features of a TVG model. In sharp contrast to 3D CNNs, we show that TVP allows us to effectively co-train vision encoder and language encoder in a 2D TVG model and improves the performance of crossmodal feature fusion using only low-complexity sparse 2D visual features. Further, we propose a Temporal-Distance IoU (TDIoU) loss for efficient learning of TVG. Experiments on two benchmark datasets, Charades-STA and ActivityNet Captions datasets, empirically show that the proposed TVP significantly boosts the performance of 2D TVG (e.g., 9.79% improvement on Charades-STA and 30.77% improvement on ActivityNet Captions) and achieves 5x inference acceleration over TVG using 3D visual features. Codes are available at Open.Intel.
X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusion

Target speech extraction (TSE) systems are designed to extract target speech from a multi-talker mixture. The popular training objective for most prior TSE networks is to enhance reconstruction performance of extracted speech waveform. However, it has been reported that a TSE system delivers high reconstruction performance may still suffer low-quality experience problems in practice. One such experience problem is wrong speaker extraction (called speaker confusion, SC), which leads to strong negative experience and hampers effective conversations. To mitigate the imperative SC issue, we reformulate the training objective and propose two novel loss schemes that explore the metric of reconstruction improvement performance defined at small chunk-level and leverage the metric associated distribution information. Both loss schemes aim to encourage a TSE network to pay attention to those SC chunks based on the said distribution information. On this basis, we present X-SepFormer, an end-to-end TSE model with proposed loss schemes and a backbone of SepFormer. Experimental results on the benchmark WSJ0-2mix dataset validate the effectiveness of our proposals, showing consistent improvements on SC errors (by 14.8% relative). Moreover, with SI-SDRi of 19.4 dB and PESQ of 3.81, our best system significantly outperforms the current SOTA systems and offers the top TSE results reported till date on the WSJ0-2mix.
DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation

Monocular depth estimation is a challenging task that predicts the pixel-wise depth from a single 2D image. Current methods typically model this problem as a regression or classification task. We propose DiffusionDepth, a new approach that reformulates monocular depth estimation as a denoising diffusion process. It learns an iterative denoising process to `denoise' random depth distribution into a depth map with the guidance of monocular visual conditions. The process is performed in the latent space encoded by a dedicated depth encoder and decoder. Instead of diffusing ground truth (GT) depth, the model learns to reverse the process of diffusing the refined depth of itself into random depth distribution. This self-diffusion formulation overcomes the difficulty of applying generative models to sparse GT depth scenarios. The proposed approach benefits this task by refining depth estimation step by step, which is superior for generating accurate and highly detailed depth maps. Experimental results on KITTI and NYU-Depth-V2 datasets suggest that a simple yet efficient diffusion approach could reach state-of-the-art performance in both indoor and outdoor scenarios with acceptable inference time.
Exploiting Contextual Structure to Generate Useful Auxiliary Tasks

Reinforcement learning requires interaction with an environment, which is expensive for robots. This constraint necessitates approaches that work with limited environmental interaction by maximizing the reuse of previous experiences. We propose an approach that maximizes experience reuse while learning to solve a given task by generating and simultaneously learning useful auxiliary tasks. To generate these tasks, we construct an abstract temporal logic representation of the given task and leverage large language models to generate context-aware object embeddings that facilitate object replacements. Counterfactual reasoning and off-policy methods allow us to simultaneously learn these auxiliary tasks while solving the given target task. We combine these insights into a novel framework for multitask reinforcement learning and experimentally show that our generated auxiliary tasks share similar underlying exploration requirements as the given task, thereby maximizing the utility of directed exploration. Our approach allows agents to automatically learn additional useful policies without extra environment interaction.
Multi-Stage Coarse-to-Fine Contrastive Learning for Conversation Intent Induction

Intent recognition is critical for task-oriented dialogue systems. However, for emerging domains and new services, it is difficult to accurately identify the key intent of a conversation due to time-consuming data annotation and comparatively poor model transferability. Therefore, the automatic induction of dialogue intention is very important for intelligent dialogue systems. This paper presents our solution to Track 2 of Intent Induction from Conversations for Task-Oriented Dialogue at the Eleventh Dialogue System Technology Challenge (DSTC11). The essence of intention clustering lies in distinguishing the representation of different dialogue utterances. The key to automatic intention induction is that, for any given set of new data, the sentence representation obtained by the model can be well distinguished from different labels. Therefore, we propose a multi-stage coarse-to-fine contrastive learning model training scheme including unsupervised contrastive learning pre-training, supervised contrastive learning pre-training, and fine-tuning with joint contrastive learning and clustering to obtain a better dialogue utterance representation model for the clustering task. In the released DSTC11 Track 2 evaluation results, our proposed system ranked first on both of the two subtasks of this Track.
Toward Unsupervised Realistic Visual Question Answering

The problem of realistic VQA (RVQA), where a model has to reject unanswerable questions (UQs) and answer answerable ones (AQs), is studied. We first point out 2 drawbacks in current RVQA research, where (1) datasets contain too many unchallenging UQs and (2) a large number of annotated UQs are required for training. To resolve the first drawback, we propose a new testing dataset, RGQA, which combines AQs from an existing VQA dataset with around 29K human-annotated UQs. These UQs consist of both fine-grained and coarse-grained image-question pairs generated with 2 approaches: CLIP-based and Perturbation-based. To address the second drawback, we introduce an unsupervised training approach. This combines pseudo UQs obtained by randomly pairing images and questions, with an RoI Mixup procedure to generate more fine-grained pseudo UQs, and model ensembling to regularize model confidence. Experiments show that using pseudo UQs significantly outperforms RVQA baselines. RoI Mixup and model ensembling further increase the gain. Finally, human evaluation reveals a performance gap between humans and models, showing that more RVQA research is needed.
Distortion-Disentangled Contrastive Learning

Self-supervised learning is well known for its remarkable performance in representation learning and various downstream computer vision tasks. Recently, Positive-pair-Only Contrastive Learning (POCL) has achieved reliable performance without the need to construct positive-negative training sets. It reduces memory requirements by lessening the dependency on the batch size. The POCL method typically uses a single loss function to extract the distortion invariant representation (DIR) which describes the proximity of positive-pair representations affected by different distortions. This loss function implicitly enables the model to filter out or ignore the distortion variant representation (DVR) affected by different distortions. However, existing POCL methods do not explicitly enforce the disentanglement and exploitation of the actually valuable DVR. In addition, these POCL methods have been observed to be sensitive to augmentation strategies. To address these limitations, we propose a novel POCL framework named Distortion-Disentangled Contrastive Learning (DDCL) and a Distortion-Disentangled Loss (DDL). Our approach is the first to explicitly disentangle and exploit the DVR inside the model and feature stream to improve the overall representation utilization efficiency, robustness and representation ability. Experiments carried out demonstrate the superiority of our framework to Barlow Twins and Simsiam in terms of convergence, representation quality, and robustness on several benchmark datasets.
The Nexus between Capital Flight and Economic Growth. A Panel Data Investigation of the Mediating Role of Inflation (Kenya: 1986-2021)

Purpose: This paper investigates whether inflation moderated the relationship between capital flight and economic growth in Kenya. The study attempts to answer one question. Has the effect of capital flight on economic growth been influenced significantly by the inflation rate in Kenya?&#x0D; Design/methodology/approach – The panel data for the variables under study, collected from World Investment Reports published by World Bank, are analyzed using feasible generalized least squares method to examine the relationship between the dependent and explanatory variables over the period 1987–2018. The interaction effect has been studied to examine the growth impact of FDI in presence of host country characteristics.&#x0D; Findings – The study found out that foreign portfolio investment outflows, profit repatriations, and inflation rates had no significant effects on economic growth when the interaction variable for inflation was included in the model. Outward foreign direct investment had a significant effect on economic growth when inflation is included in the model as an interaction variable with a positive coefficient indicating they had a positive significant relationship. The interaction variable had a negative but insignificant coefficient indicating that inflation rate did not moderate the relationship between capital flight and economic growth. From the empirical findings, we can infer that capital flight did not constrain resources and subsequently did not affect economic growth negatively. We also find that inflation rate cannot be used to control or moderate the effect of capital flight on the economy and Kenya should not pursue policies geared towards using inflation to moderate this relationship.&#x0D; Originality/value – The present study provides for the first time comprehensive empirical evidence on the relationship between inflation, economic growth, and the increased rate of capital outflows in Kenya. Therefore, this study would be of prime importance for policymakers.&#x0D; Originality/value – Very few studies have been conducted to examine the nexus between inflation, capital flight and economic growth in Kenyans financial markets. Assessing the interaction of these variables in Kenya and their impact on economic growth by use of panel data methodology is an original contribution of this paper toward the existing body of knowledge.
Orientation-based classification of sit-to-stand activity using Artificial Neural Network in real-time

Human activity recognition is highly required to develop an assistive technology. This study proposes the use of Artificial Neural Network (ANN) to classify Sit to Stand (STS) activity based on the sensor's orientation angle. There are four main phases in this research which are Sit Phase, Flexion Phase, Extension Phase, and Stabilization Phase (Stand). Human activity recognition is highly required to develop an assistive technology. STS activity is an important movement for every human being despite the inability of certain age groups to perform this movement due to weakened muscle function. The limited information from previous on the difficult phases experienced by the subjects to perform STS causes the development process of assistive devices slower. Our solution can classify those phases in real-time using the angles on Korpus Sterni (chest) and Tibia (calf) to gather information on which phase is difficult to be performed. It manages to gather and process the sensor data on application with approximately 3 seconds delay, resulting in the extension phase being a difficult phase to classify. A dataset of 32,000 samples was obtained from 8 subjects consisting of 6 subjects aged 20-30 years and 2 subjects aged 40-50 years. After experimenting and testing the performance of the ANN architecture, the neural network architecture consisted of 4 input nodes, 4 hidden layers (93-69-89-76) with appropriate hyperparameters, and 4 output layers. The training accuracy and testing accuracy reached 86% and 72% respectively.
Accumulation of hydrocarbons and some heavy metal contents on sediments and plants from crude oil polluted mangrove ecosystem in Okrika, Nigeria

Composite soil sediments samples were randomly collected from three polluted sites in Okrika local Government Area of Rivers State. Old Isakka (01), Imonitanbie (1M) and AgiahiAma (AA). The samples were investigated to examine the effects of crude oil pollution and heavy metals status of soil sediments and plant tissue samples from mangrove ecosystems.The soil sediments were collected at a distance of 0, 20, 40 meters and a control sample taken from distance of 140 meters away from the polluted sites near the river with a spade from each of the locations. Six samples were taken from each location (four soil sediments and two plant samples) totaling twelve sediments and six plant samples. The samples were analyzed for: Total hydrocarbon contents (THC), and heavy metals (Pb, Cd and Zn) using conventional methods of analysis. Result revealed a significant (P&lt;0.05) increase in THC in polluted samples over control in the three locations studied. The concentration of THC in the locations decrease in order of 01&gt;1M&gt;AA, the pollution decreased with increase in distance. The three locations were highly polluted with hydrocarbon thus posing a threat to aquatic organisms and human that consume the sea food. THC exceeded the alert and intervention limit for sensitive and less sensitive soils. Significant (P&lt;0.05) difference was observed in Cd concentration in control (D) over the polluted (A, B, C) in all the locations meaning that crude oil may not be responsible for the Cd in the sediments. Cadmium concentration was below the permissible limit for soil. Pb was significantly higher in polluted sediment than control in all the locations. The concentration of Pb decreases in the order of 01&gt;1M&gt;AA. Value of zinc was significantly higher in control than polluted. The order of increase was 01&gt;1M&gt;AA. Both Zinc and Pb were below the permissible limit. There was a significant (P&lt;0.05) increase in accumulation of THC, and Pb in plant tissues in polluted than control while the converse was observed in both Zn and Cd in the three locations studied. Generally, Old Isakka (01) has higher amount of pollution in both the sediments and plant tissues than Imonitanbie and Agiahi-Ama.
Developing interactive materials of writing non-literary texts based on flipbooks for secondary school students

his study aims to produce and determine the effectiveness of interactive material for writing non-literary texts based on flipbooks for class X students of MAN 2 Langkat. This research is a Research and Development (RnD) research using the ADDIE development model. There are three subjects in this development research. The first, validators consist of two material experts and two design experts. The four validators are Postgraduate Lecturers at Medan State University. Second, two teachers of Indonesian subjects. And the last are the students of class X MAN 2 Langkat totalled 96 people from three different classes, namely X MIA, X IIS, and X IIA. The data collection instruments used were in the form of a teacher needs analysis questionnaire, a student needs analysis questionnaire, a material and media expert validation sheet, a teacher validation sheet, and a student response sheet. Individual trials obtained "good" criteria with an average percentage of 88.5%. The small group trial obtained the criteria of "very good" with an average percentage of 89.8%. The wider trial obtained the criteria of "very good" with an average percentage of 89.5%. The effectiveness of the interactive material for writing non-literary texts based on flipbooks developed shows higher student learning outcomes with an average pre-test score of 73.4% with the lowest score being 62, and the highest score being 82. While the post-test average score -test is 86.7% with the lowest score is 75 and the highest score is 94. Based on these data, it can be concluded that the interactive material for writing non-literary texts based on flipbooks is effective and can improve student learning outcomes, especially in writing non-literary texts.
Implementation and evaluation of the effectiveness of dynamic object detection tools for multithreaded and distributed processing of graphical data

Modern monitoring and operational decision-making systems based on image series have become widely used for a wide variety of applications. At the stage of identification of graphic objects, it is established that the found dynamic object belongs to the class of objects of interest based on a comparative analysis of its contours with a given template. The article is the final in a series of articles devoted to theoretical, algorithmic and software components of the video analytics system. The description of the structural components of the video analytics system, the features of the implementation of the dynamic object detection module on video sequences, and the evaluation of the efficiency of the system is presented.
Digital Forensics Face Detection and Recognition

Abstract: Face recognition is a biometric system used to identify or verify a person who appear in a scene often captured by security cameras like at Airports, Offices, Universities, Banks, etc. In this project, we have built a Face detection and Recognition model over images and videos. We have used Viola Jones Algorithm to detect a face(s) in an digital image and Convolutional Neural Network to predict it, for identification or verification. We have extended this model to be able to detect and predict faces which are partially covered with glasses, sunglasses, mask and facemask. We have also used data augmentation to increase the dataset size which in result has boosted the performance of the model.
Dragonfly soft-computing approach for workload scheduling resource utilization maximization using multi-cloud platform

The execution of the real-time workload in the multi-cloud platform with the Service Level Agreement (SLA) requirements is a challenging task. Existing workload scheduling models have addressed issues related to minimizing execution time, cost, and energy with application reliability prerequisite. However, these models are not efficient in maximizing resource utilization under a multi-cloud platform. In addressing resource utilization issues, this paper presents a Workload Scheduling Resource Utilization Maximization (WS-RUM) technique for the multi-cloud platform. The WS-RUM technique leverages a multi-objective such as energy, processing efficiency, and fault-tolerant offloading mechanism employing a dragonfly soft computing algorithm. The WS-RUM improves resource utilization by minimizing both energy and processing time for real-time workload execution in comparison with existing workload execution.
StyleDiff: Attribute Comparison Between Unlabeled Datasets in Latent Disentangled Space

One major challenge in machine learning applications is coping with mismatches between the datasets used in the development and those obtained in real-world applications. These mismatches may lead to inaccurate predictions and errors, resulting in poor product quality and unreliable systems. In this study, we propose StyleDiff to inform developers of the differences between the two datasets for the steady development of machine learning systems. Using disentangled image spaces obtained from recently proposed generative models, StyleDiff compares the two datasets by focusing on attributes in the images and provides an easy-to-understand analysis of the differences between the datasets. The proposed StyleDiff performs in $O (d N\log N)$, where $N$ is the size of the datasets and $d$ is the number of attributes, enabling the application to large datasets. We demonstrate that StyleDiff accurately detects differences between datasets and presents them in an understandable format using, for example, driving scenes datasets.
Design of Efficient Point-Mass Filter with Application in Terrain Aided Navigation

This paper deals with state estimation of stochastic models with linear state dynamics, continuous or discrete in time. The emphasis is laid on a numerical solution to the state prediction by the time-update step of the grid-point-based point-mass filter (PMF), which is the most computationally demanding part of the PMF algorithm. A novel efficient PMF (ePMF) estimator, unifying continuous and discrete, approaches is proposed, designed, and discussed. By numerical illustrations, it is shown, that the proposed ePMF can lead to a time complexity reduction that exceeds 99.9% without compromising accuracy. The MATLAB code of the ePMF is released with this paper.
Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecasting

Multi-person pose forecasting remains a challenging problem, especially in modeling fine-grained human body interaction in complex crowd scenarios. Existing methods typically represent the whole pose sequence as a temporal series, yet overlook interactive influences among people based on skeletal body parts. In this paper, we propose a novel Trajectory-Aware Body Interaction Transformer (TBIFormer) for multi-person pose forecasting via effectively modeling body part interactions. Specifically, we construct a Temporal Body Partition Module that transforms all the pose sequences into a Multi-Person Body-Part sequence to retain spatial and temporal information based on body semantics. Then, we devise a Social Body Interaction Self-Attention (SBI-MSA) module, utilizing the transformed sequence to learn body part dynamics for inter- and intra-individual interactions. Furthermore, different from prior Euclidean distance-based spatial encodings, we present a novel and efficient Trajectory-Aware Relative Position Encoding for SBI-MSA to offer discriminative spatial information and additional interactive clues. On both short- and long-term horizons, we empirically evaluate our framework on CMU-Mocap, MuPoTS-3D as well as synthesized datasets (6 ~ 10 persons), and demonstrate that our method greatly outperforms the state-of-the-art methods. Code will be made publicly available upon acceptance.