Reviewerly/exHarmony huggingface dataset:
Analyzing the Needs of ICT Job Market in Jordan Using a Text Mining Approach

In demanding economical periods, observing ICT job market needs arises as a responsibility of employers, job seekers, and academic institutions. The novelty of this paper resides in the goal of bridging the gap between the ICT job market and academic institutions, and the technique that is used to delve into a variety of job postings using an efficient and improved research base. More than 300 job postings in Jordan were collected in 2020 and investigated using a latent semantic analysis approach. The results revealed that the most needed vacancies are web and mobile applications development, which are already covered by the traditional programs of the Jordanian academic institutions. However, the results also implied that the ICT job market is still nowhere close to realizing the potential of cybersecurity, data analytics, business intelligence, and big data skills which poses demands on the academic institutions and the industry.
Towards Practical and Efficient Long Video Summary

Recently, video summarization (VS) techniques are widely used to alleviate huge processing pressure brought by numerous long videos. However, it is hard to summarize long videos efficiently since processing hundreds of frames is still time-consuming. In this paper, we find that the Kernel Temporal Segmentation (KTS) method designed for detecting the shot boundaries in SOTA VS methods is time-consuming while handling long videos. To address this issue, we propose the Distribution-based KTS (D-KTS) by fully considering the characteristic of shot length distribution. Furthermore, we propose the Hash-based Adaptive Frame Selection (HAFS) to improve the system performance by fully taking advantage of the temporal locality of long videos. Our experiments present that the proposed D-KTS is 92.70% faster and takes up 90.08% less memory than the baseline KTS method on average.
Progressive-Granularity Retrieval Via Hierarchical Feature Alignment for Person Re-Identification

Person re-identification (re-ID) aims to match pedestrian images from non-overlapping cameras. It is a challenging task because of the feature misalignment problem caused by occlusion. In this paper, inspired by the coarse-to-fine nature of human perception, we propose a novel Progressive-Granularity Retrieval (PGR) method to tackle this issue. Specifically, (i) we define instance-level, part-level and pixel-level features for an image. PGR learns these features by a single feature extractor to capture hierarchical clues in the image. (ii) These features are inherently related but different in perceptual granularity, and they can provide complementary information. For each type of feature, we propose a corresponding similarity metric to achieve hierarchical feature alignment. (iii) In training, we learn the model end-to-end. In inference, a progressive retrieval strategy is introduced to efficiently aggregate the complementary information provided by these features. Extensive experiments on three bench-marks of both occluded and holistic-body re-ID tasks show the effectiveness of the proposed method. Especially, our method significantly outperforms state-of-the-art by 4.5% Rank-1 score on the challenging Occluded-Duke dataset.
Category-Adaptive Domain Adaptation for Semantic Segmentation

Unsupervised domain adaptation (UDA) becomes more and more popular in tackling real-world problems without ground truths of the target domain. Though tedious annotation work is not required, UDA unavoidably faces two problems: 1) how to narrow the domain discrepancy to boost the transferring performance; 2) how to improve the pseudo annotation producing mechanism for self-supervised learning (SSL). In this paper, we focus on UDA for semantic segmentation tasks. Firstly, we introduce adversarial learning into style gap bridging mechanism to keep the style information from two domains in a similar space. Secondly, to keep the balance of pseudo labels on each category, we propose a category-adaptive threshold mechanism to choose category-wise pseudo labels for SSL. The experiments are conducted using GTA5 as the source domain, Cityscapes as the target domain. The results show that our model outperforms the state-of-the-arts with a noticeable gain on cross-domain adaptation tasks.
Attribute-Conditioned Face Swapping Network for Low-Resolution Images

Deep learning based face swapping technologies have opened new frontiers for entertainment industries while pose novel threats to identity security. Applying face swapping to real-world products, as well as defending against its misuse, rely on the capacity to generate high quality face swapped images from realistic scenarios where high resolution images are hard to come by. To this end, we need to address the drawbacks of existing methods, especially on their lacking on maintaining the detail attributes and their dependency on high resolution images as inputs. In this paper, we propose a novel Attribute-Conditioned Face Swapping Network (AFSNet) to preserve attributes and handle low resolution images. Specifically, we use an Image Enhancement Network (IEN) to restore high resolution images from low resolution images and a Face Exchange Module (FEM) to swap the faces. In the FEM, we improve the fidelities of the generated images by using a novel multi-domain feature fusion module (MDFFM) to integrate the identity feature, context feature, IEN feature, and attribute vector to obtain the final image. We also design an attribute transfer loss to promote the consistency of the attributes such as beards and youth between the source and swapped images. The experiments demonstrate our method’s superior performance compared with the state-of-the-art methods.
Multiple Temporal Context Embedding Networks for Unsupervised time Series Anomaly Detection

Unsupervised anomaly detection for time series signals is challenging, due to the imbalanced distribution of data and the lack of ground-truth labels. Current methods on this topic are mainly based on deep neural networks, which are optimized by heuristic constraints or empirical priors. However, various patterns of anomalous data, especially those lasting for varying periods, are hard to be captured by plain networks. To tackle this problem, we propose a multiple temporal context embedding method. The core of our method is to construct a unified representation of the multiple temporal contexts of data, which is achieved by learning a set of base features to reconstruct the hidden features within existing anomaly detection networks. The proposed method can be implemented as a convenient plug-in module, and be combined with various network architectures, such as autoencoders and graph neural networks. Extensive experiments on multiple datasets demonstrate that the proposed method can boost the performance of baseline networks significantly.
Transductive Clip with Class-Conditional Contrastive Learning

Inspired by the remarkable zero-shot generalization capacity of vision-language pre-trained model, we seek to leverage the supervision from CLIP model to alleviate the burden of data labeling. However, such supervision inevitably contains the label noise, which significantly degrades the discriminative power of the classification model. In this work, we propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. Firstly, a class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels and boost the tolerance to noisy labels. Secondly, ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels. This framework can reduce the impact of noisy labels from CLIP model effectively by combining both techniques. Experiments on multiple benchmark datasets demonstrate the substantial improvements over other state-of-the-art methods.
Advin: Automatically Discovering Novel Domains and Intents from User Text Utterances

Recognizing the intents and domains of users’ spoken and written language is a key component of Natural Language Understanding (NLU) systems. Real applications however encounter dynamic, rapidly evolving environments with newly emerging intents and domains, for which no labeled data or prior information is available. For such a setting, we propose a novel framework, ADVIN, to automatically discover novel domains and intents from large volumes of unlabeled text. We first employ an open classification model to discriminate all utterances potentially consisting of a novel intent. Next, we train a deep learning model with a pairwise margin loss function and knowledge transfer, to discover multiple latent intent categories in an unsupervised manner. We finally form a hierarchical intent-domain taxonomy by linking mutually related novel intents into novel domains. ADVIN significantly outperforms strong baselines on four benchmark datasets, and data from a real-world voice agent.
Investigating Sequence-Level Normalisation For CTC-Like End-to-End ASR

End-to-end Automatic Speech Recognition (E2E ASR) significantly simplifies the training process of an ASR model. Connectionist Temporal Classification (CTC) is one of the most popular methods for E2E ASR training. Implicitly, CTC has a unique topology which is very useful for sequence modelling. However, we find that by changing to another topology, we can make it even more effective. In this paper, we propose a new CTC-like method, for E2E ASR training, by modifying the topology of original CTC, so that the well-known abuse of the blank label in CTC can be resolved theoretically. As we change the topology, a normalisation term is necessary, which makes the form of the final loss function similar to Maximum Mutual Information (MMI); we hence name our method MMI-CTC. In addition to maximising the posterior probability of the target sequence, the normalisation enables models to explicitly minimise the probability of competing hypothesis at the word sequence level. Our experimental results show that MMI-CTC is more efficient than CTC, and that the normalisation is essential for sequence training.
CLIPCAM: A Simple Baseline For Zero-Shot Text-Guided Object And Action Localization

The key for the contemporary deep learning-based object and action localization algorithms to work is the large-scale annotated data. However, in real-world scenarios, since there are infinite amounts of unlabeled data beyond the categories of publicly available datasets, it is not only time- and manpower-consuming to annotate all the data but also requires a lot of computational resources to train the detectors. To address these issues, we show a simple and reliable baseline that can be easily obtained and work directly for the zero-shot text-guided object and action localization tasks without introducing additional training costs by using Grad-CAM, the widely used class visual saliency map generator, with the help of the recently released Contrastive Language-Image Pre-Training (CLIP) model by OpenAI, which is trained contrastively using the dataset of 400 million image-sentence pairs with rich cross-modal information between text semantics and image appearances. With extensive experiments on the Open Images and HICO-DET datasets, the results demonstrate the effectiveness of the proposed approach for the text-guided unseen object and action localization tasks for images.
Investigating Needs of Academic Support Using Personalized Learning System to Enhance Academic Success Among Fresh Year Student

Stepping into higher education, students need good readiness and study skills to be able to adapt to increasingly complex academic demands. This study aims to investigate student needs for skills and academic support that can be provided through a personalized learning system model platform. This study uses a descriptive research design with the sample population being freshmen in 2021 in Malang City. Sampling using random sampling technique with data collection through online questionnaires and analyzed descriptively. The results showed the problems faced by students related to writing scientific papers, academic anxiety, and low self-regulation in learning. Based on these conditions, new students need support and learning skills to achieve academic success. The learning skills needed by students include the ability to overcome academic anxiety, self-regulated learning and writing scientific papers. Students really need a learning support platform in the form of an application based on a personalized learning system. The learning media chosen by students are videos and e-books.
Solar Irradiance Forecasting using Bayesian Optimization based Machine Learning Algorithm to Determine the Optimal Size of a Residential PV System

This work presents a new method for determining the optimum size of a Photovoltaic (PV) system for meeting a residential load in an unelectrified area. To evaluate the optimum power size of the PV system, the proposed work takes into consideration of the forecasted global horizontal irradiance (GHI) values determined using a Bayesian optimization based regression tree (BORT) machine learning algorithm. At each iteration, the prediction error of the regression tree is reduced by tuning the hyperparameter (leaf size) using the Bayesian optimization algorithm which results in higher prediction accuracy. Based on the forecasted GHI, the optimal power generation of the PV system is estimated. Furthermore, the effectiveness of Bayesian optimization has been compared with grid search and random search methods. The proposed forecasting process is implemented and verified using Matlab -Regression learner toolbox. The statistical results revealed that the BORT algorithm estimates GHI with a mean square error of 2% and 5% lesser than that obtained in grid search and the random search respectively.
Probabilistic Fine-Grained Urban Flow Inference with Normalizing Flows

Fine-grained urban flow inference (FUFI) aims at enhancing the resolution of traffic flow, which plays an important role in intelligent traffic management. Existing FUFI methods are mainly based on techniques from image super-resolution (SR) models, which cannot fully capture the influence of external factors and face the ill-posed problem in SR tasks. In this paper, we propose UFI-Flow – Urban Flow Inference via normalizing Flow, a novel model for addressing the FUFI problem in a principled manner by using a single probabilistic loss. UFI-Flow therefore directly accounts for the ill-posed nature of the problem and learns spatial correlations between urban flow maps. In addition, an augmented distribution fusion mechanism is further proposed to reinforce the influence of external factors in the joint distribution inference. We conduct comprehensive experiments on real-world datasets to show the superiority of the proposed model compared to the state-of-the-art baseline approaches.
Price Prediction of House using KNN based Lasso and Ridge Model

Getting a house of our wishes within our budget in a residential area of our customization is quite a tedious process. In order to overcome this, we have developed a model to get a house of our interest with religious belief and budget this Implemented model is of linear regression and k nearest neighbor's algorithm with gradient descent optimization to make an optimal model for predicting house prices using the dataset. Performed feature engineering and selection using lasso and ridge penalties to eliminate features that had little or no impact on the residual sum of squares error. Then exposes Jupyter notebook cells as REST Endpoints to make predictions with new information. Finally, we are trying to send an email alert to the concerned user to give an alert of the house price
Detail Generation and Fusion Networks for Image Inpainting

Recent image inpainting methods based on end-to-end models have achieved great success with the help of generative adversarial training and structure generation. However, the generated conditional structure priors cannot support the model to reconstruct finer texture. This is mainly because these models lack good texture prior knowledge, which leads to their limited ability to generate finer texture. In this paper, we propose a novel detail generation and fusion network (DGFNet) to strengthen the generation of texture details for image inpainting, which includes a dual-stream texture generation network and a multi-scale difference perception fusion network. The dual-stream texture generation network can explicitly model the missing texture information and generate a texture map to compensate for the coarse result produced by the parallel network. Furthermore, to merge two different kinds of information effectively, a fusion network based on the differential perception fusion module (DPFM) is introduced for multi-scale perception fusion in feature level. Extensive qualitative and quantitative experiments on the benchmark dataset show that the proposed DGFNet achieves state-of-the-art performance.
A Multitask Learning Framework for Speaker Change Detection with Content Information from Unsupervised Speech Decomposition

Speaker Change Detection (SCD) is a task of determining the time boundaries between speech segments of different speakers. SCD system can be applied to many tasks, such as speaker diarization, speaker tracking, and transcribing audio with multiple speakers. Recent advancements in deep learning lead to approaches that can directly detect the speaker change points from audio data at the frame-level based on neural network models. These approaches may be further improved by utilizing speaker information in the training data, and utilizing content information extracted in an unsupervised manner. This work proposes a novel framework for the SCD task, which utilizes a multitask learning architecture to leverage speaker information during the training stage, and adds the content information extracted from an unsupervised speech decomposition model to help detect the speaker change points. Experiment results show that the architecture of multitask learning with speaker information can improve the performance of SCD, and adding content information extracted from unsupervised speech decomposition model can further improve the performance. To the best of our knowledge, this work outperforms the state-of-the-art SCD results [1] on the AMI dataset.
End-to-End ASR-Enhanced Neural Network for Alzheimer’s Disease Diagnosis

This paper presents an approach to Alzheimer’s disease (AD) diagnosis from spontaneous speech using an end-to-end ASR-enhanced neural network. Under the condition that only audio data are provided and accurate transcripts are unavailable, this paper proposes a system that can analyze utterances to differentiate between AD patients, healthy controls, and individuals with mild cognitive impairment. The ASR-enhanced model comprises automatic speech recognition (ASR) with an encoder-decoder structure and the encoder followed by an AD classification network. The encoder takes a Mel spectrogram as input and transforms it into high-level acoustic features that correlate with AD. The classification network then maps intermediate acoustic features to three categories. In the training phase, the AD classification and speech recognition tasks are optimized simultaneously. Experimental results obtained from an AD recognition dataset of Chinese spontaneous speech <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> illustrate the effectiveness of integrating ASR into AD diagnosis in an end-to-end manner. Further, our model has low dependency on accurate ASR transcripts. This work achieved accuracy scores of 89.1% and 82.6% for long and short utterance tracks, respectively.
An Exploration of Hubert with Large Number of Cluster Units and Model Assessment Using Bayesian Information Criterion

Self-supervised learning (SSL) has become one of the most important technologies to realize spoken dialogue systems for languages that do not have much audio data and its transcription available. Speech representation models are one of the keys to achieving this, and have been actively studied in recent years. Among them, Hidden-Unit BERT (HuBERT) has shown promising results in automatic speech recognition (ASR) tasks. However, previous studies have investigated with limited iterations and cluster units. We explore HuBERT with larger numbers of clusters and iterations in order to obtain better speech representation. Furthermore, we introduce the Bayesian Information Criterion (BIC) as the performance measure of the model. Experimental results show that our model achieves the best performance in 5 out of 8 scores in the 4 metrics for the Zero Resource Speech 2021 task. It also outperforms the HuBERT BASE model trained with 960-hour LibriSpeech (LS) even though our model is only trained with 100-hour LS. In addition, we report that BIC is useful as a clue for determining the appropriate number of clusters to improve performance on phonetic, lexical, and syntactic metrics. Finally, we show that these findings are also effective for the ASR task.
Teaching CNNs to Mimic Human Visual Cognitive Process &amp; Regularise Texture-Shape Bias

Recent experiments in computer vision demonstrate texture bias as the primary reason for supreme results in models employing Convolutional Neural Networks (CNNs), conflicting with early works claiming that these networks identify objects using shape. It is believed that the cost function forces the CNN to take a greedy approach and develop a proclivity for local information like texture to increase accuracy, thus failing to explore any global statistics. We propose CognitiveCNN, a new intuitive architecture, inspired from feature integration theory in psychology to utilise human-interpretable feature like shape, texture, edges etc. to reconstruct, and classify the image. We define novel metrics to quantify the "relevance" of "abstract information" present in these modalities using attention maps. We further introduce a regularisation method which ensures that each modality like shape, texture etc. gets proportionate influence in a given task, as it does for reconstruction; and perform experiments to show the resulting boost in accuracy and robustness, be-sides imparting explainability to these CNNs for achieving superior performance in object recognition.
Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrieval

We show that supervised neural information retrieval (IR) models are prone to learning sparse attention patterns over passage tokens, which can result in key phrases including named entities receiving low attention weights, eventually leading to model under-performance. Using a novel targeted synthetic data generation method that identifies poorly attended entities and conditions the generation episodes on those, we teach neural IR to attend more uniformly and robustly to all entities in a given passage. On two public IR benchmarks, we empirically show that the proposed method helps improve both the model's attention patterns and retrieval performance, including in zero-shot settings.