Reviewerly/exHarmony huggingface dataset:
FedCM: A Real-time Contribution Measurement Method for Participants in Federated Learning

Federated Learning (FL) creates an ecosystem for multiple agents to collaborate on building models with data privacy consideration. The method for contribution measurement of each agent in the FL system is critical for fair credits allocation but few are proposed. In this paper, we develop a real-time contribution measurement method FedCM that is simple but powerful. The method defines the impact of each agent, comprehensively considers the current round and the previous round to obtain the contribution rate of each agent with attention aggregation. Moreover, FedCM updates contribution every round, which enable it to perform in real-time. Real-time is not considered by the existing approaches, but it is critical for FL systems to allocate computing power, communication resources, etc. Compared to the state-of-the-art method, the experimental results show that FedCM is more sensitive to data quantity and data quality under the premise of real-time. Furthermore, we developed federated learning open-source software based on FedCM. The software has been applied to identify COVID-19 based on medical images.
Joint Feature Selection and Extraction With Sparse Unsupervised Projection

Feature selection and feature extraction, in the field of data dimensionality reduction, are the two main strategies. Nevertheless, each of these two strategies has its own advantages and disadvantages. The features chosen by feature selection method have complete physical meaning. However, feature selection cannot reveal the implicit structural information of the samples. In this article, the methods proposed by us combine both feature selection and feature extraction, called joint feature selection and extraction with sparse unsupervised projection (SUP) and graph optimization SUP (GOSUP). A constraint on the number of nonzero rows of the projection matrix is added, which ensures the sparsity of the projection matrix, and only the features corresponding to the nonzero rows of the projection matrix are selected for the feature extraction procedure. We invoke a newly proposed algorithm to tackle this constrained optimization problem. A new concept of "purification matrix" is invented, the use of which could better eliminate meaningless information of samples in subspace. The performance on several datasets verifies the effectiveness of the proposed method for data dimensionality reduction.
CEGFNet: Common Extraction and Gate Fusion Network for Scene Parsing of Remote Sensing Images

Scene parsing of high spatial resolution (HSR) remote sensing images has achieved notable progress in recent years by the adoption of convolutional neural networks. However, for scene parsing of multimodal remote sensing images, effectively integrating complementary information remains challenging. For instance, the decrease in feature map resolution through a neural network causes loss of spatial information, likely leading to blurred object boundaries and misclassification of small objects. In addition, object scales on a remote sensing image vary substantially, undermining the parsing performance. To solve these problems, we propose an end-to-end common extraction and gate fusion network (CEGFNet) to capture both high-level semantic features and low-level spatial details for scene parsing of remote sensing images. Specifically, we introduce a gate fusion module to extract complementary features from spectral data and digital surface model data. A gate mechanism removes redundant features in the data stream and extracts complementary features that improve multimodal feature fusion. In addition, a global context module and a multilayer aggregation decoder handle scale variations between objects and the loss of spatial details due to downsampling, respectively. The proposed CEGFNet was quantitatively evaluated on benchmark scene parsing datasets containing HSR remote sensing images, and it achieved state-of-the-art performance.
Dynamic Graph Construction for Improving Diversity of Recommendation

The diversity of recommendation has attracted a lot of attention in recommender systems due to its ability to improve user experience. Most of the diversified recommendation tasks usually exploit user-item interaction records for mining user explicit preferences, while rarely explore the user-item non-interaction records. For diversified recommendations, however, the neglected non-interaction records are especially important for capturing users’ potential interests to improve the diversity of recommendation. Moreover, the majority of diversified recommendation methods run in two stages: first optimizing the users and items embeddings by relevance, then generating the diversified items list by post-processing methods. These methods are not end-to-end thus can hardly reach global optimum. To solve above limitations, we propose an end-to-end Dynamic Diversified Graph framework (DDGraph) which constructs the user-item graph dynamically based on the users and items embeddings. Technically, we initialize a user-item interaction graph and dynamically update the graph by selecting a set of diverse items for each user and building links between the items and user. The selection of diverse items can be achieved by different candidate selection operators. Specifically, we design a Quantile Progressive Candidate Selection (QPCS) operator based on the latent space division. To the best of our knowledge, our method is the first to diversify recommendation results by dynamic end-to-end graph construction and the QPCS has a higher computational efficiency than other operators. Extensive experiments on the benchmark dataset illustrate the effectiveness and superiority of the DDGraph framework.
Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation

Robots deployed in many real-world settings need to be able to acquire new skills and solve new tasks over time. Prior works on planning with skills often make assumptions on the structure of skills and tasks, such as subgoal skills, shared skill implementations, or task-specific plan skeletons, which limit adaptation to new skills and tasks. By contrast, we propose doing task planning by jointly searching in the space of parameterized skills using high-level skill effect models learned in simulation. We use an iterative training procedure to efficiently generate relevant data to train such models. Our approach allows flexible skill parameterizations and task specifications to facilitate lifelong learning in general-purpose domains. Experiments demonstrate the ability of our planner to integrate new skills in a lifelong manner, finding new task strategies with lower costs in both train and test tasks. We additionally show that our method can transfer to the real world without further fine-tuning.
Integrating N-Gram Features into Pre-trained Model: A Novel Ensemble Model for Multi-target Stance Detection

Multi-target stance detection in tweets aims to detect the stance of given texts towards a specific target entity. Most existing models on stance detection consider word embedding as input, however, recent developments pointed out that it would be beneficial to incorporate feature-based information appropriately. Motivated by the strong performance of the pre-trained models in many Natural Language Processing field, and n-gram features that have been proved to be effective in prior competition, we present a novel combination module to obtain both advantages. This paper has proposed a pre-trained model integrated with n-gram features module (PMINFM) to better utilize multi-scale feature representation information and semantic features. Then connect it to a Bidirectional Long Short-Term Memory networks with target-specific attention mechanism. The experimental results show that our proposed model outperforms other baseline models in the SemEval-2016 stance detection dataset and achieves state-of-the-art performance.
EPMDA-FED: Efficient and Privacy-Preserving Multidimensional Data Aggregation Scheme With Fast Error Detection in Smart Grid

Smart grids bring advantages of reliability and high efficiency by real-time communication technologies in contrast to the traditional grids. However, smart grids also raise concerns about privacy and security for the individual fine-grained information collection. In order to guarantee privacy and security in the grids, we propose an efficient and privacy-preserving multidimensional data aggregation scheme without a third trusted party and supporting fast error detection, named EPMDA-FED, in the article. First, we adopt a Chinese Remainder Theorem (CRT) to pack multidimensional data and encrypt the processed data using the keys generated by the negotiation among users and the control center (CC). Second, our scheme is efficient for encryption without high-cost additive homomorphic public-key encryption (PKE) scheme, such as the Paillier cryptosystem and supporting batch verification with fast error detection. Our proposed error detection algorithm is quite efficient with sublinear computational complexity. Besides, through security analysis, EPMDA-FED is semantically secure against collusion attack and the consistency of negotiated key, authenticity, and data integrity of the users’ reports are guaranteed. Finally, performance evaluation shows EPMDA-FED is more efficient than the existing competing approaches in terms of computational and communication overheads.
Influence of inhomogeneous magnetic field on the qutrit teleportation via XXZ Heisenberg chain under intrinsic decoherence

We study qutrit teleportation through a qutritxxzchain, in the presence of intrinsic decoherence and a non-homogeneous magnetic field. We study the effects of intrinsic phase change, magnetic field and entanglement of the initial state of the channel. It is observed that while the intrinsic phase change and the non-homogeneity of the magnetic field have adverse effects on the teleportation fidelity, the entanglement of the initial state of the channel enhances the latter. Moreover, the intrinsic decoherence may remove the ripples from the time curve that is delivered by the Schrödinger channel.
Analysis of Ontology Quality Dimensions, Criteria and Metrics

Ontology quality assessment needs to be performed across the ontology development life cycle to ensure that the ontology being modeled meets the intended purpose. To this end, a set of quality criteria and metrics provides a basis to assess the quality with respect to the quality requirements. However, the existing criteria and metrics defined in the literature so far are messy and vague. Thus, it is difficult to determine what set of criteria and measures would be applicable to assess the quality of an ontology for the intended purpose. Moreover, there are no well-accepted methodologies for ontology quality assessment as the way it is in the software engineering discipline. Therefore, a comprehensive review was performed to identify the existing contribution on ontology quality criteria and metrics. As a result, it was identified that the existing criteria can be classified under five dimensions namely syntactic, structural, semantic, pragmatic, and social. Moreover, a matrix with ontology levels, approaches, and criteria/metrics was presented to guide the researchers when they perform a quality assessment.
Regulatory Considerations on Centralized Aspects of DeFi Managed by DAOs

This paper focuses on the centralized governance mechanisms of decentralized finance (DeFi) projects managed by Distributed Autonomous Organizations (DAOs) and discusses regulatory considerations. Unlike highly decentralized ecosystems such as Bitcoin, the degree of decentralization varies among DeFi projects. Centralized aspects such as concentrated ownership of governance tokens and admin keys have significant implications on their governance. Concerns include decision-making concentration risk and poor alignment of interests among stakeholders. From a regulatory viewpoint, centralized aspects could make it easier for regulators to impose requirements and therefore increase compliance costs. This might drive the DeFi community to seek further decentralization to avoid regulatory burdens. We conclude that the DeFi ecosystem should learn from the experience of both Internet governance as a partially decentralized system and from traditional corporate governance.
Self-Supervised Representation Learning for Videos by Segmenting via Sampling Rate Order Prediction

Self-supervised representation learning for videos has been very attractive recently because these methods exploit the information inherently obtained from the video itself instead of annotated labels that is quite time-consuming. However, existing methods ignore the importance of global observation while performing spatio-temporal transformation perception, which highly limits the expression capabilities of the video representation. This paper proposes a novel pretext task that combines the temporal information perception of the video with the motion amplitude perception of moving objects to learn the spatio-temporal representation of the video. Specifically, given a video clip containing several video segments, each video segment is sampled by different sampling rates and the order of video segments is disrupted. Then, the network is used to regress the sampling rate of each video segment and classify the order of input video segments. In the pre-training stage, the network can learn rich spatio-temporal semantic information where content-related contrastive learning is introduced to make the learned video representation more discriminative. To alleviate the appearance dependency caused by contrastive learning, we design a novel and robust vector similarity measurement approach, which can take feature alignment into consideration. Moreover, a view synthesis framework is proposed to further improve the performance of contrastive learning by automatically generating reasonable transformed views. We conduct benchmark experiments with several 3D backbone networks on two datasets. The results show that our proposed method outperforms the existing state-of-the-art methods across the three backbones on two downstream tasks of human action recognition and video retrieval.
Talk to Me on My Level – Linguistic Alignment for Chatbots

Digital companions and conversational agents are becoming increasingly popular in our everyday lives. Natural language interfaces play an important role in ubiquitous computing: voice assistants are used to control smart home devices and smartphone applications; chatbots serve as an interface to solve tasks and acquire information easily. However, misunderstandings due to non-standard language, expressions that serve social functions without conveying information, or a lack of situational awareness still pose problems for these interfaces. Humans are able to prevent or repair communication failures by imitating their conversation partner's lexical choices, sentence structures, and overall language style; a mechanism known as Linguistic Alignment. In this paper we present different strategies to easily integrate an alignment effect in natural language interfaces. We implemented a chatbot that imitates alignment and tested it in an online user study with 75 participants. Our results show that alignment helps to decrease user frustration and perceived task workload.
MovieCuts: A New Dataset and Benchmark for Cut Type Recognition

Understanding movies and their structural patterns is a crucial task in decoding the craft of video editing. While previous works have developed tools for general analysis, such as detecting characters or recognizing cinematography properties at the shot level, less effort has been devoted to understanding the most basic video edit, the Cut. This paper introduces the Cut type recognition task, which requires modeling multi-modal information. To ignite research in this new task, we construct a large-scale dataset called MovieCuts, which contains 173, 967 video clips labeled with ten cut types defined by professionals in the movie industry. We benchmark a set of audio-visual approaches, including some dealing with the problem's multi-modal nature. Our best model achieves $$47.7\%$$ mAP, which suggests that the task is challenging and that attaining highly accurate Cut type recognition is an open research problem. Advances in automatic Cut-type recognition can unleash new experiences in the video editing industry, such as movie analysis for education, video re-editing, virtual cinematography, machine-assisted trailer generation, machine-assisted video editing, among others. Our data and code are publicly available: https://github.com/PardoAlejo/MovieCuts .
Disentangling 3D/4D Facial Affect Recognition With Faster Multi-View Transformer

In this paper, we propose MiT: a novel multi-view transformer model 1 for 3D/4D facial affect recognition.MiT incorporates patch and position embeddings from various patches of multi-views and uses them for learning various facial muscle movements to showcase an effective recognition performance.We also propose a multi-view loss function that is not only gradient-friendly, and hence speeds up the gradient computation during back-propagation, but it also leverages the correlation associated with the underlying facial patterns among multiviews.Additionally, we offer multi-view weights that are trainable and learnable, and help substantially in training.Finally, we equip our model with distributed performance for faster learning and computational convenience.With the help of extensive experiments, we show that our model outperform the existing methods on widely-used datasets for 3D/4D FER.
Full UHD 360-Degree Video Dataset and Modeling of Rate-Distortion Characteristics and Head Movement Navigation

We investigate the rate-distortion (R-D) characteristics of full ultra-high definition (UHD) 360° videos and capture corresponding head movement navigation data of virtual reality (VR) headsets. We use the navigation data to analyze how users explore the 360° look-around panorama for such content and formulate related statistical models. The developed R-D characteristics and modeling capture the spatiotemporal encoding efficiency of the content at multiple scales and can be exploited to enable higher operational efficiency in key use cases. The high quality expectations for next generation immersive media necessitate the understanding of these intrinsic navigation and content characteristics of full UHD 360° videos.
Joint Feature Optimization and Fusion for Compressed Action Recognition

Recent methods including CoViAR and DMC-Net provide a new paradigm for action recognition since they are directly targeted at compressed videos (e.g., MPEG4 files). It avoids the cumbersome decoding procedure of traditional methods, and leverages the pre-encoded motion vectors and residuals in compressed videos to complete recognition efficiently. However, motion vectors and residuals are noisy, sparse and highly correlated information, which cannot be effectively exploited by plain and separated networks. To tackle these issues, we propose a joint feature optimization and fusion framework that better utilizes motion vectors and residuals in the following three aspects. (i) We model the feature optimization problem as a reconstruction process that represents features by a set of bases, and propose a joint feature optimization module that extracts bases in the both modalities. (ii) A low-rank non-local attention module, which combines the non-local operation with the low-rank constraint, is proposed to tackle the noise and sparsity problem during the feature reconstruction process. (iii) A lightweight feature fusion module and a self-adaptive knowledge distillation method are introduced, which use motion vectors and residuals to generate predictions similar to those from networks with optical flows. With these proposed components embedded in a baseline network, the proposed network not only achieves the state-of-the-art performance on HMDB-51 and UCF-101, but also maintains its advantage in computational complexity.
Lattice-Based HRA-secure Attribute-Based Proxy Re-Encryption in Standard Model

Proxy re-encryption (PRE), introduced by Blaze, Bleumer, and Strauss at EUROCRYPT 98, offers delegation of decryption rights, i.e., it securely enables the re-encryption of ciphertexts from one key to another, without relying on trusted parties. PRE allows a semi-trusted third party termed as a “proxy” to securely divert ciphertexts of a user (delegator) to another user (delegatee) without revealing any information about the underlying messages to the proxy. Attribute-based proxy re-encryption (ABPRE) generalizes PRE by allowing such transformation of ciphertext under an access-policy into another ciphertext under a new access policy. Such a primitive facilitates fine-grained secure sharing of encrypted data in the cloud. In order to capture the application goals of PRE, the security model of (Attribute-based) PRE evolves over the decades. There are two well-established notions of security for (Attribute-based) proxy re-encryption schemes: security under chosen-plaintext attacks (CPA) and security under chosen-ciphertext attacks (CCA). Both definitions aim to address the security that the delegator enjoys against both proxy and delegatee. Recently, at PKC 19, Cohen points out that CPA security guarantees much less security against delegatee than was previously understood. In particular, CPA security does not prevent delegatee from learning delegator’s secret key after receiving a single honestly re-encrypted ciphertext. To circumvent this issue, Cohen proposes security against honest re-encryption attacks (HRA) to strengthen CPA security that better captures the goals of PRE, and shows that two existing proxy re-encryption schemes are HRA-secure, one of them is quantum-safe, which is constructed from fully homomorphic encryption scheme (FHE). In this work, we advance the studies on HRA-secure PRE for the ABE setting. We first formalize the definition of HRA-secure Key-Policy ABPRE (\(\mathsf {KP\hbox {-}ABPRE}\)) and propose a construction, which is quantum-safe and secure in the standard model based on the hardness of the \(\mathsf {LWE}\). As an important consequence, we have the first quantum-safe HRA-secure Identity-based PRE. Moreover, the underlying PRE of the proposed \(\mathsf {KP\hbox {-}ABPRE}\) is the first quantum-safe HRA-secure PRE without FHE.
Analysis of Benford’s Law for No-Reference Quality Assessment of Natural, Screen-Content, and Synthetic Images

With the tremendous growth and usage of digital images, no-reference image quality assessment is becoming increasingly important. This paper presents in-depth analysis of Benford’s law inspired first digit distribution feature vectors for no-reference quality assessment of natural, screen-content, and synthetic images in various viewpoints. Benford’s law makes a prediction for the probability distribution of first digits in natural datasets. It has been applied among others for detecting fraudulent income tax returns, detecting scientific fraud, election forensics, and image forensics. In particular, our analysis is based on first digit distributions in multiple domains (wavelet coefficients, DCT coefficients, singular values, etc.) as feature vectors and the extracted features are mapped onto image quality scores. Extensive experiments have been carried out on seven large image quality benchmark databases. It has been demonstrated that first digit distributions are quality-aware features, and it is possible to reach or outperform the state-of-the-art with them.