Reviewerly/exHarmony huggingface dataset:
Simple Attention Module Based Speaker Verification with Iterative Noisy Label Detection

Recently, the attention mechanism such as squeeze-and-excitation module (SE) and convolutional block attention module (CBAM) has achieved great success in deep learning-based speaker verification system. This paper introduces an alternative effective yet simple one, i.e., simple attention module (SimAM), for speaker verification. The SimAM module is a plug-and-play module without extra modal parameters. In addition, we propose a noisy label detection method to iteratively filter out the data samples with a noisy label from the training data, considering that a large-scale dataset labeled with human annotation or other automated processes may contain noisy labels. Data with the noisy label may over parameterize a deep neural network (DNN) and result in a performance drop due to the memorization effect of the DNN. Experiments are conducted on VoxCeleb dataset. The speaker verification model with SimAM achieves the 0.675% equal error rate (EER) on VoxCeleb1 original test trials. Our proposed iterative noisy label detection method further reduces the EER to 0.643%.
ESCALATE: Boosting the Efficiency of Sparse CNN Accelerator with Kernel Decomposition

The ever-growing parameter size and computation cost of Convolutional Neural Network (CNN) models hinder their deployment onto resource-constrained platforms. Network pruning techniques are proposed to remove the redundancy in CNN parameters and produce a sparse model. Sparse-aware accelerators are also proposed to reduce the computation cost and memory bandwidth requirements of inference by leveraging the model sparsity. The irregularity of sparse patterns, however, limits the efficiency of those designs. Researchers proposed to address this issue by creating a regular sparsity pattern through hardware-aware pruning algorithms. However, the pruning rate of these solutions is largely limited by the enforced sparsity patterns. This limitation motivates us to explore other compression methods beyond pruning. With two decoupled computation stages, we found that kernel decomposition could potentially take the processing of the sparse pattern off from the critical path of inference and achieve a high compression ratio without enforcing the sparse patterns. To exploit these advantages, we propose ESCALATE, an algorithm-hardware co-design approach based on kernel decomposition. At algorithm level, ESCALATE reorganizes the two computation stages of the decomposed convolution to enable a stream processing of the intermediate feature map. We proposed a hybrid quantization to exploit the different reuse frequency of each part of the decomposed weight. At architecture level, ESCALATE proposes a novel ‘Basis-First’ dataflow and its corresponding microarchitecture design to maximize the benefits brought by the decomposed convolution.
Generalized Multi-View Collaborative Subspace Clustering

In real-world applications, complete or incomplete multi-view data are common, which leads to the problem of generalized multi-view clustering. Recently, researchers attempt to learn the latent representation in the common subspace from heterogeneous data, which usually suffers from feature degeneration. Moreover, there are limited efforts on simultaneously revealing the underlying subspace structure and exploring the complementary information from incomplete multiple views. In this paper, we introduce a novel Generalized Multi-view Collaborative Subspace Clustering (GMCSC) framework to address the above issues, in which consensus subspace structure of all views and embedding subspaces for each view are jointly learned to benefit each other. Specifically, we develop a novel collaborative subspace learning strategy based on self-representation learning, which provides a brand-new way of pursuing the complete subspace structure directly from multi-view data. Furthermore, we explore complementary information by enforcing the consistency across different views and preserving the view-specific information of each view, which can alleviate the problem of feature degeneration and enhance the reasonability of using a consensus representation for multiple views. Experimental results on six benchmark datasets demonstrate that the proposed method can significantly outperform the state-of-the-art algorithms.
An SVD Values Ordering Scheme for Medical Image Watermarking

In order to increase the security of medical image sharing and transfer, we present in this work a watermarking approach to protect medical images. This approach consists of inserting hospital signature information and patient data into the medical image. The goal of our work is to integrate the watermark with as little distortion as possible to retain the medical information in the image. In this approach DWT decomposition is applied to the image which allows a very fine adjustment during the insertion. An SVD is then applied to the three subbands LL, LH and HL, which allows storing the maximum energy of the image in a minimum of singular values. A combination of the three resulting singular value matrices is then performed for watermark integration. The proposed approach ensures data integrity, confidentiality when sharing data, and robustness to several conventional attacks.
RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance

Deep learning recommendation systems must provide high quality, personalized content under strict tail-latency targets and high system loads. This paper presents RecPipe, a system to jointly optimize recommendation quality and inference performance. Central to RecPipe is decomposing recommendation models into multi-stage pipelines to maintain quality while reducing compute complexity and exposing distinct parallelism opportunities. RecPipe implements an inference scheduler to map multi-stage recommendation engines onto commodity, heterogeneous platforms (e.g., CPUs, GPUs). While the hardware-aware scheduling improves ranking efficiency, the commodity platforms suffer from many limitations requiring specialized hardware. Thus, we design RecPipeAccel (RPAccel), a custom accelerator that jointly optimizes quality, tail-latency, and system throughput. RPAccel is designed specifically to exploit the distinct design space opened via RecPipe. In particular, RPAccel processes queries in sub-batches to pipeline recommendation stages, implements dual static and dynamic embedding caches, a set of top-k filtering units, and a reconfigurable systolic array. Compared to previously proposed specialized recommendation accelerators and at iso-quality, we demonstrate that RPAccel improves latency and throughput by 3 × and 6 ×.
Stochastic Image Denoising by Sampling from the Posterior Distribution

Image denoising is a well-known and well studied problem, commonly targeting a minimization of the mean squared error (MSE) between the outcome and the original image. Unfortunately, especially for severe noise levels, such Minimum MSE (MMSE) solutions may lead to blurry output images. In this work we propose a novel stochastic denoising approach that produces viable and high perceptual quality results, while maintaining a small MSE. Our method employs Langevin dynamics that relies on a repeated application of any given MMSE denoiser, obtaining the reconstructed image by effectively sampling from the posterior distribution. Due to its stochasticity, the proposed algorithm can produce a variety of high-quality outputs for a given noisy input, all shown to be legitimate denoising results. In addition, we present an extension of our algorithm for handling the inpainting problem, recovering missing pixels while removing noise from partially given data.
Performance Evaluation of Ethereum Private and Testnet Networks Using Hyperledger Caliper

Since Bitcoin was launched, the blockchain technology and the cryptocurrencies have been in the spotlight. Ethereum, the second-generation blockchain introduced smart contracts, and many DApps have emerged due to them. Those DApps showed the feasibility of blockchain in various industries. However, even though the growth of blockchain technology, still many DApps are based on Ethereum, and supper its performance issue. Since the performance evaluation in Ethereum mainnet is almost impossible, and there are no formalized performance evaluation frameworks, it is hard to perform appropriate performance evaluation of Ethereum. Detail performance evaluations on Ethereum networks are essential for developing and operating DApps. In this paper, we use Hyperledger Caliper, an automated performance evaluation framework to evaluate an Ethereum private network, and the Ropsten testnet to overcome above problems. We evaluate the performance with a specific smart contract and analyze the results. Our evaluation results show that the Ethereum private network performs better than the Ropsten testnet, and the Ropsten testnet is unstable for performance evaluation. In addition, our results show that the performance of the transactions can differ following their content.
A Study on Pokémon GO: Exploring the Potential of Location-based Mobile Exergames in Connecting Players with Nature

Exergaming incorporates exercising into video games, with the purpose of physically engaging users in the gameplay. Location-based games have gained the attention of exergame designers as they have been able to motivate people to exercise and spend time in outdoors. Studies have shown the impact of current exergames to intervene with sedentariness by containing some elements of nature and outdoor activities. However, it is unclear how location-based mobile exergames can engage users with nature throughout playing and exercising. This paper presents insights from an exploratory self-study of experiences provided by the most popular location-based mobile game, Pokémon GO, concerning its potential to engage players in outdoors. We also provide design considerations for location-based mobile exergames to engage players in nature while becoming physically active.
Consumer responses to the use of artificial intelligence in luxury and non-luxury restaurants

There has been an ever-increasing interest in artificial intelligence (AI) in the hospitality sector. However, it is still unclear how consumers respond to products/services in hospitality industries provided by AI. Building on the theoretical framework for using AI in different services and the literature on luxury consumption, across four studies, we investigated how consumers evaluate different types of restaurants that are run by AI vs. humans. The results demonstrated that consumers evaluated food and restaurants more negatively where the service is provided by AI (Study 1). This effect was higher in luxury restaurants compared to casual dining (fast food, casual restaurants) (Studies 2 and 3). Moreover, we identified the underlying mechanism of this effect by showing that in luxury (vs. casual) restaurants, foods cooked by AI negatively influenced evaluations of three dimensions of restaurants’ quality, such as food, service, and ambience quality, decreasing the intention to visit the restaurant (Study 4). Altogether, these findings reveal some effects that AI can have in the hospitality industry and provide practical insights on how to introduce AI in restaurants.
AALpy: An Active Automata Learning Library

AALpy is an extensible open-source Python library providing efficient implementations of active automata learning algorithms for deterministic, non-deterministic, and stochastic systems. We put a special focus on the conformance testing aspect in active automata learning, as well as on an intuitive and seamlessly integrated interface for learning automata characterizing real-world reactive systems. In this manuscript, we present AALpy's core functionalities, illustrate its usage via examples, and evaluate its learning performance.
Imbalanced Source-free Domain Adaptation

Conventional Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a well-labeled source domain to an unlabeled target domain only when data from both domains is simultaneously accessible, which is challenged by the recent Source-free Domain Adaptation (SFDA). However, we notice that the performance of existing SFDA methods would be dramatically degraded by intra-domain class imbalance and inter-domain label shift. Unfortunately, class-imbalance is a common phenomenon in real-world domain adaptation applications. To address this issue, we present Imbalanced Source-free Domain Adaptation (ISFDA) in this paper. Specifically, we first train a uniformed model from the source domain, and then propose secondary label correction, curriculum sampling, plus intra-class tightening and inter-class separation to overcome the joint presence of covariate shift and label shift. Extensive experiments on three imbalanced benchmarks verify that ISFDA could perform favorably against existing UDA and SFDA methods under various conditions of class-imbalance, and outperform existing SFDA methods by over 15% in terms of per-class average accuracy on a large-scale long-tailed imbalanced dataset.
Consistency-Constancy Bi-Knowledge Learning for Pedestrian Detection in Night Surveillance

Pedestrian detection in the night surveillance is a challenging yet not largely explored task. As the success of the detector in the daytime surveillance and the convenient acquisition of all-weather data, we learn knowledge from these data to benefit pedestrian detection in night surveillance. We find two key properties of surveillance: distribution cross-time consistency and background cross-frame constancy. This paper proposes a consistency-constancy bi-knowledge learning (CCBL) for pedestrian detection in night surveillance, which is able to simultaneously achieve the night pedestrian detection's useful knowledge, coming from day and night surveillance. Firstly, based on the robustness of the existing detector in day surveillance, we obtain pedestrians' distribution in the daytime scene using the detector's detection results in the daytime scene. Based on the consistency of pedestrians' distribution during the day and night in the same scene, the pedestrian distribution from daytime is used as the consistency-knowledge for pedestrian detection in night surveillance. Secondly, the background as a constant knowledge of the surveillance scene is extractable and contributes to the division of the foreground, which contains most of the pedestrian regions and helps in pedestrian detection for night surveillance. Finally, we add bi-knowledge representation to promote each other and merge them together as the final pedestrian representation. Through extensive experiments, our CCBL significantly outperforms the state-of-the-art methods on public pedestrian detection datasets. In the NightSurveillance dataset, CCBL reduced the average missed detection rate by 3.04% compared to the existing best method.
Heterogeneous Feature Fusion and Cross-modal Alignment for Composed Image Retrieval

Composed image retrieval aims at performing image retrieval task by giving a reference image and a complementary text piece. Since composing both image and text information can accurately model the users' search intent, composed image retrieval can perform target-specific image retrieval task and be potentially applied to many scenarios such as interactive product search. However, two key challenging issues must be addressed in composed image retrieval occasion. One of them is how to fuse heterogeneous image and text piece in the query into a complementary feature space. The other is how to bridge the heterogeneous gap between text pieces in the query and images in the database. To address the issues, we propose an end-to-end framework for composed image retrieval, which consists of three key components including Multi-modal Complementary Fusion (MCF), Cross-modal Guided Pooling (CGP), and Relative Caption-aware Consistency (RCC). By incorporating MCF and CGP modules, we can fully integrate the complementary information of image and text piece in the query through multiple deep interactions and aggregate obtained local features into an embedding vector. To bridge the heterogeneous gap, we introduce the RCC constraint to align text pieces in the query and images in the database. Extensive experiments on four public benchmark datasets show that the proposed composed image retrieval framework achieves outstanding performance against the state-of-the-art methods.
Multimodal Video Summarization via Time-Aware Transformers

With the growing number of videos in video sharing platforms, how to facilitate the searching and browsing of the user-generated video has attracted intense attention by multimedia community. To help people efficiently search and browse relevant videos, summaries of videos become important. The prior works in multimodal video summarization mainly explore visual and ASR tokens as two separate sources and struggle to fuse the multimodal information for generating the summaries. However, the time information inside videos is commonly ignored. In this paper, we find that it is important to leverage the timestamps to accurately incorporate multimodal signals for the task. We propose a Time-Aware Multimodal Transformer (TAMT) with a novel short-term order-sensitive attention mechanism. The attention mechanism can attend the inputs differently based on time difference to explore the time information inherent inside video more thoroughly. As such, TAMT can fuse the different modalities better for summarizing the videos. Experiments show that our proposed approach is effective and achieves the state-of-the-art performances on both YouCookII and open-domain How2 datasets.
zkrpChain: Towards multi-party privacy-preserving data auditing for consortium blockchains based on zero-knowledge range proofs

Consortium blockchain has been widely used in different scenarios, where blockchain members demand that their uploaded data could be audited based on data range under their identities without exposing the data themselves. However, so far, current solutions only support simple (e.g., standard-range) privacy-preserving data auditing, and can not be used in multi-party scenarios. To address the problem, we propose zkrpChain, which focuses on protection of the integrity and privacy of the data uploaded by blockchain members while leaving their identities public. In zkrpChain, which is based on Hyperledger Fabric and Bulletproofs, both standard-range and arbitrary-range zero-knowledge range proofs generation and verification are supported. To improve the efficiency, the aggregation of multiple proofs and batch verification are developed. Furthermore, to broaden the application scenarios, we also develop multi-party proofs generation and verification. To support further development, we provide chaincodes, related APIs together with the client codes. Finally, we conduct experiments to evaluate the performance of zkrpChain, and the results show that as the total data size grows exponentially, the proof and verification time only grow linearly while the total on-chain storage grows logarithmically.
Video Similarity and Alignment Learning on Partial Video Copy Detection

Existing video copy detection methods generally measure video similarity based on spatial similarities between key frames, neglecting the latent similarity in temporal dimension, so that the video similarity is biased towards spatial information. There are methods modeling unified video similarity in an end-to-end way, but losing detailed partial alignment information, which causes the incapability of copy segments localization. To address the above issues, we propose the Video Similarity and Alignment Learning (VSAL) approach, which jointly models spatial similarity, temporal similarity and partial alignment. To mitigate the spatial similarity bias, we model the temporal similarity as the mask map predicted from frame-level spatial similarity, where each element indicates the probability of frame pair lying right on the partial alignments. To further localize partial copies, the step map is learned from the spatial similarity where the elements indicate extending directions of the current partial alignments on the spatial-temporal similarity map. Obtained from the mask map, the start points extend out into partial optimal alignments following instructions of the step map. With the similarity and alignment learning strategy, VSAL achieves the state-of-the-art F1-score on VCDB core dataset. Furthermore, we construct a new benchmark of partial video copy detection and localization by adding new segment-level annotations for FIVR-200k dataset, where VSAL also achieves the best performance, verifying its effectiveness in more challenging situations. Our project is publicly available at https://pvcd-vsal.github.io/vsal/.
Solar Radiation Prediction Using a Novel Hybrid Model of ARMA and NARX

The prediction of solar radiation has a significant role in several fields such as photovoltaic (PV) power production and micro grid management. The interest in solar radiation prediction is increasing nowadays so efficient prediction can greatly improve the performance of these different applications. This paper presents a novel solar radiation prediction approach which combines two models, the Auto Regressive Moving Average (ARMA) and the Nonlinear Auto Regressive with eXogenous input (NARX). This choice has been carried out in order to take the advantages of both models to produce better prediction results. The performance of the proposed hybrid model has been validated using a real database corresponding to a company located in Barcelona north. Simulation results have proven the effectiveness of this hybrid model to predict the weekly solar radiation averages. The ARMA model is suitable for small variations of solar radiation while the NARX model is appropriate for large solar radiation fluctuations.
StaResGRU-CNN with CMedLMs: A stacked residual GRU-CNN with pre-trained biomedical language models for predictive intelligence

As a task requiring strong professional experience as supports, predictive biomedical intelligence cannot be separated from the support of a large amount of external domain knowledge. By using transfer learning to obtain sufficient prior experience from massive biomedical text data, it is essential to promote the performance of specific downstream predictive and decision-making task models. This is an efficient and convenient method, but it has not been fully developed for Chinese Natural Language Processing (NLP) in the biomedical field. This study proposes a Stacked Residual Gated Recurrent Unit-Convolutional Neural Networks (StaResGRU-CNN) combined with the pre-trained language models (PLMs) for biomedical text-based predictive tasks. Exploring related paradigms in biomedical NLP based on transfer learning of external expert knowledge and comparing some Chinese and English language models. We have identified some key issues that have not been developed or those present difficulties of application in the field of Chinese biomedicine. Therefore, we also propose a series of Chinese bioMedical Language Models (CMedLMs) with detailed evaluations of downstream tasks. By using transfer learning, language models are introduced with prior knowledge to improve the performance of downstream tasks and solve specific predictive NLP tasks related to the Chinese biomedical field to serve the predictive medical system better. Additionally, a free-form text Electronic Medical Record (EMR)-based Disease Diagnosis Prediction task is proposed, which is used in the evaluation of the analyzed language models together with Clinical Named Entity Recognition, Biomedical Text Classification tasks. Our experiments prove that the introduction of biomedical knowledge in the analyzed models significantly improves their performance in the predictive biomedical NLP tasks with different granularity. And our proposed model also achieved competitive performance in these predictive intelligence tasks.