Reviewerly/exHarmony huggingface dataset:
Fair Selection of Edge Nodes to Participate in Clustered Federated Multitask Learning

Clustered federated Multitask learning is introduced as an efficient technique when data is unbalanced and distributed amongst clients in a non-independent and identically distributed manner. While a similarity metric can provide client groups with specialized models according to their data distribution, this process can be time-consuming because the server needs to capture all data distribution first from all clients to perform the correct clustering. Due to resource and time constraints at the network edge, only a fraction of devices is selected every round, necessitating the need for an efficient scheduling technique to address these issues. Thus, this paper introduces a two-phased client selection and scheduling approach to improve the convergence speed while capturing all data distributions. This approach ensures correct clustering and fairness between clients by leveraging bandwidth reuse for participants spent a longer time training their models and exploiting the heterogeneity in the devices to schedule the participants according to their delay. The server then performs the clustering depending on predetermined thresholds and stopping criteria. When a specified cluster approximates a stopping point, the server employs a greedy selection for that cluster by picking the devices with lower delay and better resources. The convergence analysis is provided, showing the relationship between the proposed scheduling approach and the convergence rate of the specialized models to obtain convergence bounds under non-i.i.d. data distribution. We carry out extensive simulations, and the results demonstrate that the proposed algorithms reduce training time and improve the convergence speed by up to 50% while equipping every user with a customized model tailored to its data distribution.
Artificial Intelligence Technologies in College English Translation Teaching

This paper explores the practical prospects for using artificial intelligence technologies in professional English-speaking translator education. At the online conference 'Translation Skills in Times of Artificial Intelligence' (DingTalk platform, January 2022), the teachers of higher education institutions in China prioritized the translator's competencies necessary for successful professional activity during the digital transformation of social and economic business relations. The educators also evaluated the demand for online services used in the education of English-Chinese interpreters. The survey results showed that the use of artificial intelligence technologies in educational practices could have a significant impact on the development of key competencies of future translators. Using a competency-based approach to interpreter training and considering the need to develop abilities, knowledge, and skills required for successful professional translation activity, the author developed the pedagogical concept of the online educational course 'Simultaneous and asynchronous translation in a digital environment.'
Explainable AI for customer segmentation in product development

AI can facilitate customer segmentation in product development. However, the black-box nature of AI often causes designers to distrust AI predictions. Explainable AI (XAI) is an emerging AI paradigm that can provide humanly understandable explanations about AI predictions. The outputs of XAI, feature-based and data-based explanations, can improve AI performance and designer trust in AI. A new framework is proposed to incorporate XAI explanations into customer segmentation. An experiment is conducted to validate the framework effectiveness. It indicates that AI performance can be enhanced by XAI, where feature-based explanations can facilitate feature selection and data-based explanations can uncover high-value datasets.
Multi-teacher knowledge distillation based on joint Guidance of Probe and Adaptive Corrector

Knowledge distillation (KD) has been widely used in model compression. But, in the current multi-teacher KD algorithms, the student can only passively acquire the knowledge of the teacher's middle layer in a single form and all teachers use identical a guiding scheme to the student. To solve these problems, this paper proposes a multi-teacher KD based on joint Guidance of Probe and Adaptive Corrector (GPAC) method. First, GPAC proposes a teacher selection strategy guided by the Linear Classifier Probe (LCP). This strategy allows the student to select better teachers in the middle layer. Teachers are evaluated using the classification accuracy detected by LCP. Then, GPAC designs an adaptive multi-teacher instruction mechanism. The mechanism uses instructional weights to emphasize the student's predicted direction and reduce the student's difficulty learning from teachers. At the same time, every teacher can formulate guiding scheme according to the Kullback-Leibler divergence loss of the student and itself. Finally, GPAC develops a multi-level mechanism for adjusting spatial attention loss. this mechanism uses a piecewise function that varies with the number of epochs to adjust the spatial attention loss. This piecewise function classifies the student' learning about spatial attention into three levels, which can efficiently use spatial attention of teachers. GPAC and the current state-of-the-art distillation methods are tested on CIFAR-10 and CIFAR-100 datasets. The experimental results demonstrate that the proposed method in this paper can obtain higher classification accuracy.
EPT-GCN: Edge propagation-based time-aware graph convolution network for POI recommendation

In location-based social networks (LBSNs), point-of-interest (POI) recommendation systems help users identify unvisited POIs by filtering large amounts of information. Accurate POI recommendations can effectively improve user satisfaction and save time in finding POIs. In recent years, the graph convolution network (GCN) technique, which enhances the representational ability of neural networks by learning the embeddings of users and items, has been widely adopted in recommendation systems to improve accuracy. Combining GCN with various information, such as time and geographical information, can further improve recommendation performance. However, existing GCN-based techniques simply adopt time information by modeling users' check-in sequences, which is insufficient and ignores users' time-based high-order connectivity. Note that time-based high-order connectivity refers to the relationship between indirect neighbors with similar preferences in the same time slot. In this paper, we propose a new time-aware GCN model to extract rich collaborative signals contained in time information. Our work is the first to divide user check-ins into multiple subgraphs, i.e., time slots, based on time information. We further propose an edge propagation module to adjust edge affiliation, where edges represent check-ins, to propagate user's time-based preference to multiple time slots. The propagation module is based on an unsupervised learning algorithm and does not require additional ground-truth labels. Experimental results confirm that our method outperforms state-of-the-art GCN models in all baselines, improving Recall@5 from 0.0803 to 0.0874 (8.84%) on the Gowalla dataset and from 0.0360 to 0.0388 (7.78%) on the New York dataset. The proposed subgraph mining technique and novel edge-based propagation module have high scalability and can be applied to other subgraph construction models.
Combining global receptive field and spatial spectral information for single-image hyperspectral super-resolution

Single-image hyperspectral super-resolution has poorer reconstruction performance in the spatial dimension than fused-image hyperspectral super-resolution due to the lack of auxiliary images. Some studies have attempted to use 3D convolution to explore hidden features between spatial spectra to enhance spatial details. However, either 2D or 3D convolution, the obtained receptive fields are limited, ignoring the effect of global spatial information on hyperspectral image reconstruction, and cannot be used for long-range dependent modeling. Therefore, we make the first attempt to combine Transformer with 3D convolution in single-image hyperspectral super-resolution and propose a 3D convolution and Transformer hyperspectral super-resolution (3D-THSR) network, which explores the hidden information between space and spectra while obtaining the global receptive field of space. Specifically, the Transformer module is used for feature extraction to enhance the learning ability of global spatial information and long-distance features. In addition, the 3D convolution module is embedded in the Transformer module to extract the information between different spectral bands by fusing the spectral and spatial dimensions. Finally, we design to train the network by using three loss functions to reduce the distortion spectrum and ensure the spectral band purity. Compared with other single-image hyperspectral methods by six hyperspectral evaluation metrics, spatial detail image, spectral error line map, and ablation study, it is proved that the proposed method achieves better hyperspectral super-resolution reconstruction performance.
Active learning using Generative Adversarial Networks for improving generalization and avoiding distractor points

In supervised computer vision tasks, convolutional neural networks (CNNs) have demonstrated superiority over alternative methods. However, training and validating these models requires large-scale labeled datasets, which are generated using specific expert knowledge and are expensive. With the limitation of a limited labeling budget, active learning is a promising approach for generating labeled datasets. The conventional methods used for active learning only consider a pool of unlabeled data that are relevant to the target task. However, in several real-world tasks such as web search results, this pool includes several data points that are irrelevant to the target task, which are called distractor points. These tend to largely degrade the effectiveness of active learning. To address this problem, we propose a method to avoid distractor points. The proposed method utilizes criteria that incorporates existing active learning methods and an effective training strategy using Generative Adversarial Networks (GANs) to eliminate distractor points from the area including the selection criteria. As a result, the proposed method successfully divides the data points into data points utilized in active learning and distractor points to improve generalization and decrease the accuracy degradation of the system. In the benchmark verification process, we show that the proposed method is effective when the number of distractor points in the considered datasets is small or large.
PDCSN: A partition density clustering with self-adaptive neighborhoods

Density-based clustering can discover convex and non-convex clusters without specifying the number of clusters. However, its ability to handle clusters with heterogeneous densities is limited. Although various variants solve this problem to some extent, they are still powerless for adjacent clusters with similar densities. Furthermore, the clustering performance is heavily dependent on user-specified parameters. This paper proposes a partition density clustering with self-adaptive neighborhoods (PDCSN). For the parameter dependence problem, a self-adaptive approach based on the natural neighborhood is designed. This approach utilizes the differences and intrinsic properties of sample density distributions to automatically find the optimal neighborhood size λ. A partitioning strategy based on mutual λ-nearest neighbors-connectedness is proposed to distinguish clusters with large varying densities. Moreover, a core sample search approach based on shared similarity and density heterogeneity is proposed to identify adjacent clusters with similar densities. A series of experiments on 9 synthetic, 10 real-world, and 2 image datasets demonstrate that PDCSN outperforms several famous clustering algorithms.
Artificial Intelligence–Enabled Gesture‐Language‐Recognition Feedback System Using Strain‐Sensor‐Arrays‐Based Smart Glove

Wearable smart glove of gesture language provides a novel strategy for the hearing‐impaired people to commutate with the world. Current commercialized solutions of gesture language are limited by the full extent of human interaction beyond operation dexterity, sensory feedback, and the huge cost of fabrication. Herein, a low‐cost, high‐efficient gesture‐language‐recognition feedback system combined with the strain‐sensor arrays and machine‐learning technology is proposed. The strain‐sensor arrays integrated with 3D‐printed glove can extract both spatial and temporal information about the finger's movement. The smart glove achieves gesture‐language recognition using machine learning with an accuracy of over 99%. Integrating with multidimensional manipulation, visual feedback and artificial intelligence (AI)‐based gesture‐language recognition, the smart system can accurately recognize complex gestures and provide real‐time feedback to users. The smart glove system can not only provide an efficient way for hearing‐impaired persons to communicate with the outside world, but also benefit industries in multiple fields such as entertainment, home healthcare, sports training, and the medical industry.
Evaluating Trustworthiness and Racial Bias in Face Recognition APIs Using Deepfakes

We evaluate the racial bias in face recognition application programming interfaces (APIs) using real and deepfake celebrity images. We use deepfake generation methods to introduce small, imperceptible changes to the real images to shift the racial class of predictions, showing how deepfake images exacerbated racial bias in web-based face recognition APIs.
Face deidentification with controllable privacy protection

Privacy protection has become a crucial concern in today’s digital age. Particularly sensitive here are facial images, which typically not only reveal a person’s identity, but also other sensitive personal information. To address this problem, various face deidentification techniques have been presented in the literature. These techniques try to remove or obscure personal information from facial images while still preserving their usefulness for further analysis. While a considerable amount of work has been proposed on face deidentification, most state-of-the-art solutions still suffer from various drawbacks, and (a) deidentify only a narrow facial area, leaving potentially important contextual information unprotected, (b) modify facial images to such degrees, that image naturalness and facial diversity is suffering in the deidentify images, (c) offer no flexibility in the level of privacy protection ensured, leading to suboptimal deployment in various applications, and (d) often offer an unsatisfactory trade-off between the ability to obscure identity information, quality and naturalness of the deidentified images, and sufficient utility preservation. In this paper, we address these shortcomings with a novel controllable face deidentification technique that balances image quality, identity protection, and data utility for further analysis. The proposed approach utilizes a powerful generative model (StyleGAN2), multiple auxiliary classification models, and carefully designed constraints to guide the deidentification process. The approach is validated across four diverse datasets (CelebA-HQ, RaFD, XM2VTS, AffectNet) and in comparison to 7 state-of-the-art competitors. The results of the experiments demonstrate that the proposed solution leads to: (a) a considerable level of identity protection, (b) valuable preservation of data utility, (c) sufficient diversity among the deidentified faces, and (d) encouraging overall performance.
Distillation from Heterogeneous Models for Top-K Recommendation

Recent recommender systems have shown remarkable performance by using an ensemble of heterogeneous models. However, it is exceedingly costly because it requires resources and inference latency proportional to the number of models, which remains the bottleneck for production. Our work aims to transfer the ensemble knowledge of heterogeneous teachers to a lightweight student model using knowledge distillation (KD), to reduce the huge inference costs while retaining high accuracy. Through an empirical study, we find that the efficacy of distillation severely drops when transferring knowledge from heterogeneous teachers. Nevertheless, we show that an important signal to ease the difficulty can be obtained from the teacher’s training trajectory. This paper proposes a new KD framework, named HetComp, that guides the student model by transferring easy-to-hard sequences of knowledge generated from the teachers’ trajectories. To provide guidance according to the student’s learning state, HetComp uses dynamic knowledge construction to provide progressively difficult ranking knowledge and adaptive knowledge transfer to gradually transfer finer-grained ranking information. Our comprehensive experiments show that HetComp significantly improves the distillation quality and the generalization of the student model.
pFedPrompt: Learning Personalized Prompt for Vision-Language Models in Federated Learning

Pre-trained vision-language models like CLIP show great potential in learning representations that capture latent characteristics of users. A recently proposed method called Contextual Optimization (CoOp) introduces the concept of training prompt for adapting pre-trained vision-language models. Given the lightweight nature of this method, researchers have migrated the paradigm from centralized to decentralized system to innovate the collaborative training framework of Federated Learning (FL). However, current prompt training in FL mainly focuses on modeling user consensus and lacks the adaptation to user characteristics, leaving the personalization of prompt largely under-explored. Researches over the past few years have applied personalized FL (pFL) approaches to customizing models for heterogeneous users. Unfortunately, we find that with the variation of modality and training behavior, directly applying the pFL methods to prompt training leads to insufficient personalization and performance. To bridge the gap, we present pFedPrompt, which leverages the unique advantage of multimodality in vision-language models by learning user consensus from linguistic space and adapting to user characteristics in visual space in a non-parametric manner. Through this dual collaboration, the learned prompt will be fully personalized and aligned to the user’s local characteristics. We conduct extensive experiments across various datasets under the FL setting with statistical heterogeneity. The results demonstrate the superiority of our pFedPrompt against the alternative approaches with robust performance.
Modeling Temporal Positive and Negative Excitation for Sequential Recommendation

Sequential recommendation aims to predict the next item which interests users via modeling their interest in items over time. Most of the existing works on sequential recommendation model users’ dynamic interest in specific items while overlooking users’ static interest revealed by some static attribute information of items, e.g., category, brand. Moreover, existing works often only consider the positive excitation of a user’s historical interactions on his/her next choice on candidate items while ignoring the commonly existing negative excitation, resulting in insufficiently modeling dynamic interest. The overlook of static interest and negative excitation will lead to incomplete interest modeling and thus impedes the recommendation performance. To this end, in this paper, we propose modeling both static interest and negative excitation for dynamic interest to further improve the recommendation performance. Accordingly, we design a novel Static-Dynamic Interest Learning (SDIL) framework featured with a novel Temporal Positive and Negative Excitation Modeling (TPNE) module for accurate sequential recommendation. TPNE is specially designed for comprehensively modeling dynamic interest based on temporal positive and negative excitation learning. Extensive experiments on three real-world datasets show that SDIL can effectively capture both static and dynamic interest and outperforms state-of-the-art baselines.
SISSI: An Architecture for Semantic Interoperable Self-Sovereign Identity-based Access Control on the Web

We present an architecture for authentication and authorization on the Web that is based on the Self-Sovereign Identity paradigm. Using our architecture, we aim to achieve semantic interoperability across different approaches to SSI. We build on the underlying RDF data model of the W3C’s recommendation for Verifiable Credentials and specify semantic access control rules using SHACL. Our communication protocol for an authorization process is based on Decentralised Identifiers and extends the Hyperledger Aries Present Proof protocol. We propose a modular architecture that allows for flexible extension, e. g., for supporting more signature schemes or Decentralised Identifier Methods. For evaluation, we implemented a Proof-of-Concept: We show that a Web-based approach to SSI outperfoms a blockchain-based approach to SSI in terms of End-to-End execution time.
Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer

Knowledge graphs (KG) are essential background knowledge providers in many tasks. When designing models for KG-related tasks, one of the key tasks is to devise the Knowledge Representation and Fusion (KRF) module that learns the representation of elements from KGs and fuses them with task representations. While due to the difference of KGs and perspectives to be considered during fusion across tasks, duplicate and ad hoc KRF modules design are conducted among tasks. In this paper, we propose a novel knowledge graph pretraining model KGTransformer that could serve as a uniform KRF module in diverse KG-related tasks. We pretrain KGTransformer with three self-supervised tasks with sampled sub-graphs as input. For utilization, we propose a general prompt-tuning mechanism regarding task data as a triple prompt to allow flexible interactions between task KGs and task data. We evaluate pretrained KGTransformer on three tasks, triple classification, zero-shot image classification, and question answering. KGTransformer consistently achieves better results than specifically designed task models. Through experiments, we justify that the pretrained KGTransformer could be used off the shelf as a general and effective KRF module across KG-related tasks. The code and datasets are available at https://github.com/zjukg/KGTransformer.