Reviewerly/exHarmony huggingface dataset:
Finest Magic Cloth or a Naked Emperor? The SKQuest Data Set on Software Metrics for Improving Transparency and Quality

Software development has a problem with transparency/visibility. As an intangible product, software and its intermediate development results are hard to see or touch. Customers of custom software have difficulties checking progress, and risk coming out with costly but low-quality software. In the space domain with its often expensive and one-of-a-kind devices, which are developed in complex multitier supply chains, the risk is even greater. This paper presents the SKQuest data set. It contains the completed responses with 190 variables from an empirical study with over 100 software experts. The data set covers distinct aspects of measuring metrics and transparency in software projects. To show what information lies in the data set, the paper investigates, and affirms, from different perspectives, the following questions: Is transparency a problem in software development projects? Is there a desire for more transparency in projects? Can metrics contribute to improving the situation? Moreover, it attempts to replicate the results of an earlier study. The main contribution of this paper is, however, the SKQuest data set that is published with this paper in CSV formatas. It is a tool that enables systematic investigations of software metrics and allows research on how they can improve the efficiency of the software lifecycle, not limited to, but particularly with respect to transparency. Consequently, the paper may serve as a starting point for future research avenues in academia and industry and help to improve existing and future standards in software development.
RØROS: Building a Responsive Online Recommender System via Meta-Gradients Updating

In the era of information explosion, users of online services are urgently waiting for timely and effective recommendations. In this paper, we present the first study on the responsiveness aspect of recommender system and present Responsive Online RecOmmender System (RØROS) based on Meta-Gradients Update (MGU) techniques, which helps improve the recommendation quality for both existing and new users when the system only observes a limited number of in-coming interactions. Technically, we propose a new sampling strategy to enable the recommender model to update itself in the unit of user-centered interaction graphs, which helps balance the responsiveness to different types of users. For model updating, we propose an MGU-based strategy to effectively capture users’ current short-term preference, which hence improve RØROS’s responsiveness to users. Mean-while, RØROS also maintains users’ long-term preference with a preference memory module. Extensive experiments and case studies on standard benchmarks validate the responsiveness of our novel recommender system design, which we hope would foster future studies in this new direction.
Runtime Prediction of Machine Learning Algorithms in Automl Systems

In this paper we introduce a metalearning-based methodology for predicting the training runtime of various machine learning algorithms. This prediction is important for automated machine learning (AutoML) systems because they search by training and evaluating a large number of machine learning models in order to identify the best model for a given dataset. Our approach identifies the main factors that impact the runtime performance of state of the art algorithms used in AutoML systems and can be used to enhance their performance in resource-constrained settings.
Encoder-Decoder Graph Convolutional Network for Automatic Timed-Up-and-Go and Sit-to-Stand Segmentation

Vision-based action segmentation is an important tool in human movement analysis. In this work, we present a novel Encoder-Decoder Graph Convolutional Network (ED-GCN) to perform auto-segmentation on two widely accepted clinical tests for human mobility and balance assessment: the "Timed-Up-and-Go" (TUG) test and the "Sit-to-Stand" (STS) test. For STS, we perform a fine-grained segmentation that further segments the stand up and sit down actions into more sub-phases. To the best of our knowledge, this is the first work that analyzes such subtle segmentation with biomedical significance. We also propose two novel metrics for action segmentation, which overcome some key drawbacks in the popular F1 and Edit scores. Experiment shows that our network has superior performance over state-of-the-art action segmentation networks in TUG and STS segmentation.
RGB-D Based Pose-Invariant Face Recognition Via Attention Decomposition Module

Face recognition has recently achieved remarkable performance with the help of deep learning networks, but there is still a domain gap between frontal and profile face recognition. Generally speaking, we utilize pose-invariant face recognition methods or incorporate additional depth information to handle pose variations. However, huge backbones or multiple models are often used in RGB-D face recognition methods, which makes them hard to be applied in edge devices. In this work, we propose a RGB-D based pose-invariant face recognition model which is light enough to meet the demands of edge devices. First, we use the attention decomposition mechanism to decompose the mixed feature maps into pose- and identity-related features layer by layer. Second, a continuously indexed domain adaptation and a multi-task training framework are applied to our proposed model to decorrelate these two components. Third, we design our proposed model by embedded convolution neural network (eCNN) architecture to reduce parameters and operations. Finally, we evaluate our proposed method on the public KincetFaceDB. The Rank-1 recognition rate of our proposed method reaches 98.23% on KincetFaceDB, which is 0.13% superior to that of [1]. The parameters of our proposed eCNN model is 0.58M, which is 98.47% lower than that of [1].
Learning Unbiased Rewards with Mutual Information in Adversarial Imitation Learning

A powerful method for automated decision systems is Adversarial Imitation Learning (AIL). It is based on a generative adversarial framework that alternately optimizes a generator (learner) and a discriminator (reward function). In the popular mind, a high-accuracy discriminator results in informative rewards, thus AIL must balance the performance between a generator and a discriminator. However, we find that the AIL reward function is biased and also results in informative rewards. Thus, we theoretically analyze the bias in the AIL reward function and find that balancing the performance of a generator and a discriminator is not necessary when we recover an unbiased reward function. Further, we propose a mutual information based auxiliary reward function. Experiments on continuous control tasks indicate that MI-GAIL is able to address the bias problem of the AIL reward function and further improve sample efficiency and training stability compared with up-to-date algorithms.
CPD-GAN: Cascaded Pyramid Deformation GAN for Pose Transfer

Pose-guided person image generation aims to synthesize person images in arbitrary poses. This task requires to perform spatial deformation on source images. Existing work often failed to transfer complex textures to generated images well. To solve this problem, we propose a novel network for this task. The network is called Cascaded Pyramid Deformation GAN(CPD-GAN), which can achieve more realistic results and conform more to the target person. In the extraction sub-network, the multi-scale feature modulation(MFM) blocks are proposed. A MFM block can fuse features at different scales into single scale feature. And in the genration sub-network, the dynamic transferring fusion(DTF) blocks are proposed to perform dynamic deformation on features from extraction sub-network and a cascaded pyramids structure is adopted to improve the quality of resulted complex textures. Experiments prove that our method performs better on pose transfer than other methods and utilizes each part of the network effectively.
Supercm: Revisiting Clustering for Semi-Supervised Learning

The development of semi-supervised learning (SSL) has in recent years largely focused on the development of new consistency regularization or entropy minimization approaches, often resulting in models with complex training strategies to obtain the desired results. In this work, we instead propose a novel approach that explicitly incorporates the underlying clustering assumption in SSL through extending a recently proposed differentiable clustering module. Leveraging annotated data to guide the cluster centroids results in a simple end-to-end trainable deep SSL approach. We demonstrate that the proposed model improves the performance over the supervised-only baseline and show that our framework can be used in conjunction with other SSL methods to further boost their performance.
Learned Video Coding with Motion Compensation Mixture Model

Learned video coding employs explicit motion compensation (MC) with neural networks to predict the original frame from its reference frame and to compress its residual from the predicted frame, where neural networks are optimized with rate-distortion trade-offs. However, good predictions are hard to find or even do not exist due to fast motions, dis-occlusions, and coding errors of the reference frame. To avoid the problem of carrying false edges/details caused by inaccurate optical flow in the predicted frame to the residual, we propose a dynamic mixture of explicit and implicit motion compensations, where implicitness means that the encoding and decoding of the original frame are conditioned on the predicted frame in pixel and latent domains, respectively. The proposed mixture model saves up to 30% bitrate over the baseline and achieves state-of-the-art performance.
Double Compression Detection Based on the De-Blocking Filtering of HEVC Videos

Instead of detecting whether the whole video sequence is double compressed, a frame-level detection result can provide more precise information for video forensic tasks, such as locate tamper point and restore compression history, et al. But the research on frame-level double compression detection is still in its infancy. Therefore we aim to provide a frame-level detection method for HEVC videos in this paper. The relocated I(RI) frame belongs to different GOP groups from its reference frame at the first compression and may cause more severe blocking effects than other types of P frames. Hence, this paper proposes an algorithm based on the de-blocking filtering feature mode to detect RI frames in the double compressed HEVC videos with shifted GOP structure. Firstly, the abnormal traces of the de-blocking filtering parameters, such as boundary strength, filtering switch and filtering mode, in the RI frame are analyzed. Then, the de-blocking filtering feature is constructed by mapping the different combinations of the three parameters into a single numerical value. Finally, the de-blocking filtering feature of the video clips is adopted as the input of the proposed mini_MobileViT network, which is the combination of Convolutional Neural Network (CN-N) and Transformer, to learn spatial and temporal representations to identify the RI frames. Experimental results demonstrate the advantages of the proposed algorithm in detecting RI frames in the double compressed HEVC videos. Compared with the state-of-art work He’s method, the proposed method has a 1.72% improvement in the accuracy of detecting RI frames. Compared with other traditional methods, there is a more than 10% improvement.
Zero-Shot Anomalous Sound Detection in Domestic Environments Using Large-Scale Pretrained Audio Pattern Recognition Models

Anomalous sound detection is central to audio-based surveillance and monitoring. In a domestic environment, however, the classes of sounds to be considered anomalous are situation-dependent and cannot be determined in advance. At the same time, it is not feasible to expect a demanding labeling effort from the end user. To address these problems, we present a novel zero-shot method relying on an auxiliary large-scale pretrained audio neural network in support of an unsupervised anomaly detector. The auxiliary module is tasked to generate a fingerprint for each sound occasionally registered by the user. These fingerprints are then compared with those extracted from the input audio stream, and the resulting similarity score is used to increase or reduce the sensitivity of the base detector. Experimental results on synthetic data show that the proposed method substantially improves upon the unsupervised base detector and is capable of outperforming existing few-shot learning systems developed for machine condition monitoring without involving additional training.
UNTAG: Learning Generic Features for Unsupervised Type-Agnostic Deepfake Detection

This paper introduces a novel framework for unsupervised type-agnostic deepfake detection called UNTAG. Existing methods are generally trained in a supervised manner at the classification level, focusing on detecting at most two types of forgeries; thus, limiting their generalization capability across different deepfake types. To handle that, we reformulate the deepfake detection problem as a one-class classification supported by a self-supervision mechanism. Our intuition is that by estimating the distribution of real data in a discriminative feature space, deepfakes can be detected as outliers regardless of their type. UNTAG involves two sequential steps. First, deep representations are learned based on a self-supervised pretext task focusing on manipulated regions. Second, a oneclass classifier fitted on authentic image embeddings is used to detect deepfakes. The results reported on several datasets show the effectiveness of UNTAG and the relevance of the proposed new paradigm. The code is publicly available.
Learn Topological Representation with Flexible Manifold Layer

Deep neural networks for classification are one of the most fundamental topics in machine learning. However, the widely used softmax decision layer, originating from the conventional softmax regression for multi-class classification, fails to guide the powerful feature extractor to explore the topological structure hidden in data, which limits the quality of produced representations. Therefore, we propose a flexible manifold layer for better representation learning in this paper, rather than adding some regularized losses to introduce extra mechanisms. The flexible manifold layer is inspired by SVM which maximizes the geometric margin and usually achieves better results than softmax regression. Since the potential structure is captured due to the explicit guidance of the proposed flexible manifold layer, the noise can be easily detected and filtered out via a designed automatic mechanism. In addition to passing the topological knowledge by latent representations, a knowledge distillation from the manifold layer and decision layer is developed. Extensive experiments show that the proposed model has achieved excellent performance. The code is available at https://github.com/jzh9830/DeepSVM.
Robust Multi-Object Tracking With Spatial Uncertainty

Most methods address the multi-object tracking (MOT) problem by tracking-by-detection paradigm, which tracks objects from the detected windows by associating detection boxes whose scores are higher than a given threshold. As such, the confidence score becomes the only indicator of bounding boxes when handling complicated cases, such as occlusions. However, a high confidence score cannot guarantee that the bounding box does not overlap with nearby objects, especially in crowded scenarios. In this paper, spatial uncertainty is proposed for MOT. Firstly, the statistical analysis indicates that spatial uncertainty is highly correlated to the occlusion ratio, which can better represent the level of occlusion of the detection boxes. It is measured by the proposed Sparse Tracker with Spatial Uncertainty (SSUTracker). Then, it is adopted to learn robust tracklet representation. The experimental results demonstrate that it improves overall performance. As a result, our approach achieves very competitive results on popular MOT17 and MOT20 benchmarks compared to state-of-the-art methods.
Context-Aware end-to-end ASR Using Self-Attentive Embedding and Tensor Fusion

Typical automatic speech recognition (ASR) systems are built to recognize independent utterances without using the cross-utterance context. However, the context over multiple utterances often provides useful information for the ASR task. In this work, we propose a context-aware end-to-end ASR model that injects the self-attentive context embedding into the decoder of the recurrent neural network transducer (RNN-T). We also propose a factorised 3-way tensor fusion approach to fuse the context embedding with the acoustic representations extracted from the acoustic encoder and the text representations obtained using the prediction network based on the previous subword units. Experimental results on a long-form Youtube ASR task shows that the proposed approach achieves 10.8% relative word error rate reductions.
AutoGCF: Personalized Aggregation on Neural Graph Collaborative Filtering

Graph neural networks have achieved state-of-the-art performance on collaborative filtering (NeuGCFs). The success of NeuGCFs is mainly attributed to the stacking of message aggregation layers. Recent studies have shown that the effectiveness of existing NeuGCFs largely relies on the selection of optimal aggregation steps, which makes the performance on various recommendation scenarios unsatisfactory. To tackle this, we for the first time propose a framework to achieve personalized aggregation step assignment on NeuGCF. First, each user is endowed with a learnable unit to measure the aggregation degree at each aggregation stage. Second, the aggregation degree is used to determine whether the aggregation process should stop after the current stage. Third, the learnable unit is jointly trained with the NeuGCF by bi-level optimization. Finally, we term this new framework AutoGCF, implementing it on the state-of-the-art NeuGCF model - Light-GCN. Empirical experiments on five datasets demonstrate the effectiveness of AutoGCF, and highlight that AutoGCF can adaptively choose optimal steps on different datasets.
Deep Learning-Based Stereo Camera Multi-Video Synchronization

Stereo vision is essential for many applications. Currently, the synchronization of the streams coming from two cameras is done using mostly hardware. A software-based synchronization method would reduce the cost, weight and size of the entire system and allow for more flexibility when building such systems. With this goal in mind, we present here a comparison of different deep learning-based systems and prove that some are efficient and generalizable enough for such a task. This study paves the way to a production ready software-based video synchronization system.
A Sidecar Separator Can Convert A Single-Talker Speech Recognition System to A Multi-Talker One

Although automatic speech recognition (ASR) can perform well in common non-overlapping environments, sustaining performance in multi-talker overlapping speech recognition remains challenging. Recent research revealed that ASR model’s encoder captures different levels of information with different layers – the lower layers tend to have more acoustic information, and the upper layers more linguistic. This inspires us to develop a Sidecar separator to empower a well-trained ASR model for multi-talker scenarios by separating the mixed speech embedding between two suitable layers. We experimented with a wav2vec 2.0-based ASR model with a Sidecar mounted. By freezing the parameters of the original model and training only the Sidecar (8.7 M, 8.4% of all parameters), the proposed approach outperforms the previous state-of-the-art by a large margin for the 2-speaker mixed LibriMix dataset, reaching a word error rate (WER) of 10.36%; and obtains comparable results (7.56%) for LibriSpeechMix dataset when limited training.
Boosting No-Reference Super-Resolution Image Quality Assessment with Knowledge Distillation and Extension

Deep learning (DL) based image super-resolution (SR) tech-niques have been well investigated for recent years. However, studies dedicated to SR image quality assessment (SR-IQA) have not been fully developed, which is even more difficult if pristine high-resolution (HR) images are lacking as a reference. Due to the challenge, existing widely used no-reference (NR) SR-IQA metrics (e.g., PI, NIQE, and Ma) are still far from meeting the practical requirements of providing accurate estimations which align well with human mean opinion scores (MOS). To this end, we propose a novel Knowledge Extension Super-Resolution Image Quality Assessment (KE-SR-IQA) framework to predict SR image quality by leveraging a semi-supervised knowledge distillation (KD) strategy. Concretely, we first employ a well-trained full-reference (FR) SR-IQA model as the teacher, then we perform knowledge extension (KE) by additional pseudo-labeled data to further distill a NR-student for promoting the prediction accuracy. Extensive experiments on several benchmarks validate the ef-fectiveness of our approach.
Delay-Penalized Transducer for Low-Latency Streaming ASR

In streaming automatic speech recognition (ASR), it is desirable to reduce latency as much as possible while having minimum impact on recognition accuracy. Although a few existing methods are able to achieve this goal, they are difficult to implement due to their dependency on external alignments. In this paper, we propose a simple way to penalize symbol delay in transducer model, so that we can balance the trade-off between symbol delay and accuracy for streaming models without external alignments. Specifically, our method adds a small constant times (T/2 - t), where T is the number of frames and t is the current frame, to all the non-blank log-probabilities (after normalization) that are fed into the two dimensional transducer recursion. For both streaming Conformer models and unidirectional long short-term memory (LSTM) models, experimental results show that it can significantly reduce the symbol delay with an acceptable performance degradation. Our method achieves similar delay-accuracy trade-off to the previously published FastEmit, but we believe our method is preferable because it has a better justification: it is equivalent to penalizing the average symbol delay. Our work is open-sourced and publicly available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
The effect of gold, dollar and Composite Stock Price Index on cryptocurrency

This paper aims to analyze cryptocurrency volatility by examining the effect of Gold, Dollar Index, and Composite Stock Price Index (IHSG) as independent variables and on Bitcoin and Ethereum as dependent variables. The cryptocurrency objects in this study are Bitcoin and Ethereum, which have the largest market capitalization. The data in this study used the period January 1, 2018, to December 31, 2021. This study used GARCH analysis. This study's results indicate that Bitcoin's volatility is influenced by the price of Bitcoin itself, gold, and the stock exchange index, and Ethereum and the stock exchange index influence Ethereum. This shows that the cryptocurrency market is inefficient as the prices are also affected by past prices.