OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic Link Prediction Using Graph Representation Learning with Enhanced Structure and Temporal Information
Venue: arXiv cs.SI 2023
Year: 2023

The links in many real networks are evolving with time. The task of dynamic link prediction is to use past connection histories to infer links of the network at a future time. How to effectively learn the temporal and structural pattern of the network dynamics is the key. In this paper, we propose a graph representation learning model based on enhanced structure and temporal information (GRL\_EnSAT). For structural information, we exploit a combination of a graph attention network (GAT) and a self-attention network to capture structural neighborhood. For temporal dynamics, we use a masked self-attention network to capture the dynamics in the link evolution. In this way, GRL\_EnSAT not only learns low-dimensional embedding vectors but also preserves the nonlinear dynamic feature of the evolving network. GRL\_EnSAT is evaluated on four real datasets, in which GRL\_EnSAT outperforms most advanced baselines. Benefiting from the dynamic self-attention mechanism, GRL\_EnSAT yields better performance than approaches based on recursive graph evolution modeling.
Enhancing Mapless Trajectory Prediction through Knowledge Distillation
Venue: arXiv cs.CV 2023
Year: 2023

Scene information plays a crucial role in trajectory forecasting systems for autonomous driving by providing semantic clues and constraints on potential future paths of traffic agents. Prevalent trajectory prediction techniques often take high-definition maps (HD maps) as part of the inputs to provide scene knowledge. Although HD maps offer accurate road information, they may suffer from the high cost of annotation or restrictions of law that limits their widespread use. Therefore, those methods are still expected to generate reliable prediction results in mapless scenarios. In this paper, we tackle the problem of improving the consistency of multi-modal prediction trajectories and the real road topology when map information is unavailable during the test phase. Specifically, we achieve this by training a map-based prediction teacher network on the annotated samples and transferring the knowledge to a student mapless prediction network using a two-fold knowledge distillation framework. Our solution is generalizable for common trajectory prediction networks and does not bring extra computation burden. Experimental results show that our method stably improves prediction performance in mapless mode on many widely used state-of-the-art trajectory prediction baselines, compensating for the gaps caused by the absence of HD maps. Qualitative visualization results demonstrate that our approach helps infer unseen map information.
Optimal Feed-Forward Control for Robotic Transportation of Solid and Liquid Materials via Nonprehensile Grasp
Venue: arXiv cs.RO 2023
Year: 2023

In everyday life, we often find that we can maintain an object's equilibrium on a tray by adjusting its orientation. Building upon this observation and extending the method we previously proposed to suppress sloshing in a moving vessel, this paper presents a feedforward control approach for transporting objects with a robot that are not firmly grasped but simply placed on a tray. The proposed approach combines smoothing actions and end-effector re-orientation to prevent object sliding. It can be integrated into existing robotic systems as a plug-in element between the reference trajectory generator and the robot control. To demonstrate the effectiveness of the proposed methods, particularly when dealing with unknown reference signals, we embed them in a direct teleoperation scheme. In this scheme, the user commands the robot carrying the tray by simply moving their hand in free space, with the hand's 3D position detected by a motion capture system. Furthermore, in the case of point-to-point motions, the same feedforward control, when fed with step inputs representing the desired goal position, dynamically generates the minimum-time reference trajectory that complies with velocity and acceleration constraints, thus avoiding sloshing and slipping. More information and accompanying videos can be found at https://sites.google.com/view/robotwaiter/
A Safe Genetic Algorithm Approach for Energy Efficient Federated Learning in Wireless Communication Networks
Venue: arXiv cs.NE 2023
Year: 2023

Federated Learning (FL) has emerged as a decentralized technique, where contrary to traditional centralized approaches, devices perform a model training in a collaborative manner, while preserving data privacy. Despite the existing efforts made in FL, its environmental impact is still under investigation, since several critical challenges regarding its applicability to wireless networks have been identified. Towards mitigating the carbon footprint of FL, the current work proposes a Genetic Algorithm (GA) approach, targeting the minimization of both the overall energy consumption of an FL process and any unnecessary resource utilization, by orchestrating the computational and communication resources of the involved devices, while guaranteeing a certain FL model performance target. A penalty function is introduced in the offline phase of the GA that penalizes the strategies that violate the constraints of the environment, ensuring a safe GA process. Evaluation results show the effectiveness of the proposed scheme compared to two state-of-the-art baseline solutions, achieving a decrease of up to 83% in the total energy consumption.
Weakly Supervised Scene Text Generation for Low-resource Languages
Venue: arXiv cs.CV 2023
Year: 2023

A large number of annotated training images is crucial for training successful scene text recognition models. However, collecting sufficient datasets can be a labor-intensive and costly process, particularly for low-resource languages. To address this challenge, auto-generating text data has shown promise in alleviating the problem. Unfortunately, existing scene text generation methods typically rely on a large amount of paired data, which is difficult to obtain for low-resource languages. In this paper, we propose a novel weakly supervised scene text generation method that leverages a few recognition-level labels as weak supervision. The proposed method is able to generate a large amount of scene text images with diverse backgrounds and font styles through cross-language generation. Our method disentangles the content and style features of scene text images, with the former representing textual information and the latter representing characteristics such as font, alignment, and background. To preserve the complete content structure of generated images, we introduce an integrated attention module. Furthermore, to bridge the style gap in the style of different languages, we incorporate a pre-trained font classifier. We evaluate our method using state-of-the-art scene text recognition models. Experiments demonstrate that our generated scene text significantly improves the scene text recognition accuracy and help achieve higher accuracy when complemented with other generative methods.
Let's Do a Thought Experiment: Using Counterfactuals to Improve Moral Reasoning
Venue: arXiv cs.CL 2023
Year: 2023

Language models still struggle on moral reasoning, despite their impressive performance in many other tasks. In particular, the Moral Scenarios task in MMLU (Multi-task Language Understanding) is among the worst performing tasks for many language models, including GPT-3. In this work, we propose a new prompting framework, Thought Experiments, to teach language models to do better moral reasoning using counterfactuals. Experiment results show that our framework elicits counterfactual questions and answers from the model, which in turn helps improve the accuracy on Moral Scenarios task by 9-16% compared to other zero-shot baselines. Interestingly, unlike math reasoning tasks, zero-shot Chain-of-Thought (CoT) reasoning doesn't work out of the box, and even reduces accuracy by around 4% compared to direct zero-shot. We further observed that with minimal human supervision in the form of 5 few-shot examples, the accuracy of the task can be improved to as much as 80%.
New Euclidean and Hermitian Self-Dual Cyclic Codes with Square-Root-Like Minimum Distances
Venue: arXiv cs.IT 2023
Year: 2023

Binary self-dual codes with large minimum distances, such as the extended Hamming code and the Golay code, are fascinating objects in the coding theory. They are closely related to sporadic simple groups, lattices and invariant theory. A family of binary self-dual repeated-root cyclic codes with lengths $n_i$ and minimum distances $d_i \geq \frac{1}{2} \sqrt{n_i+2}$, $n_i$ goes to the infinity for $i=1,2, \ldots$, was constructed in a paper of IEEE Trans. Inf. Theory, 2009. In this paper, we construct families of Euclidean self-dual repeated-root cyclic codes over the field ${\bf F}_{2^s}$, $s \geq 2$, with lengths $n_i$ and minimum distances at least $\sqrt{2^{s-1}n}-2^s$, where lengths $n_i$ go to the infinity. We also construct families of Hermitian self-dual repeated-root cyclic codes over the field ${\bf F}_{2^{2s}}$, $s \geq 1$, with lengths $n_i$ and minimum distances at least $\sqrt{n_i/2}$, where lengths $n_i$ go to the infinity. Our results show that Euclidean and Hermitian self-dual codes with large automorphism groups and large minimum distances can always be constructed.
A differentiable Gaussian Prototype Layer for explainable Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

We introduce a Gaussian Prototype Layer for gradient-based prototype learning and demonstrate two novel network architectures for explainable segmentation one of which relies on region proposals. Both models are evaluated on agricultural datasets. While Gaussian Mixture Models (GMMs) have been used to model latent distributions of neural networks before, they are typically fitted using the EM algorithm. Instead, the proposed prototype layer relies on gradient-based optimization and hence allows for end-to-end training. This facilitates development and allows to use the full potential of a trainable deep feature extractor. We show that it can be used as a novel building block for explainable neural networks. We employ our Gaussian Prototype Layer in (1) a model where prototypes are detected in the latent grid and (2) a model inspired by Fast-RCNN with SLIC superpixels as region proposals. The earlier achieves a similar performance as compared to the state-of-the art while the latter has the benefit of a more precise prototype localization that comes at the cost of slightly lower accuracies. By introducing a gradient-based GMM layer we combine the benefits of end-to-end training with the simplicity and theoretical foundation of GMMs which will allow to adapt existing semi-supervised learning strategies for prototypical part models in future.
Multitask Learning for Multiple Recognition Tasks: A Framework for Lower-limb Exoskeleton Robot Applications
Venue: arXiv cs.RO 2023
Year: 2023

To control the lower-limb exoskeleton robot effectively, it is essential to accurately recognize user status and environmental conditions. Previous studies have typically addressed these recognition challenges through independent models for each task, resulting in an inefficient model development process. In this study, we propose a Multitask learning approach that can address multiple recognition challenges simultaneously. This approach can enhance data efficiency by enabling knowledge sharing between each recognition model. We demonstrate the effectiveness of this approach using Gait phase recognition (GPR) and Terrain classification (TC) as examples, the most conventional recognition tasks in lower-limb exoskeleton robots. We first created a high-performing GPR model that achieved a Root mean square error (RMSE) value of 2.345 $\pm$ 0.08 and then utilized its knowledge-sharing backbone feature network to learn a TC model with an extremely limited dataset. Using a limited dataset for the TC model allows us to validate the data efficiency of our proposed Multitask learning approach. We compared the accuracy of the proposed TC model against other TC baseline models. The proposed model achieved 99.5 $\pm$ 0.044% accuracy with a limited dataset, outperforming other baseline models, demonstrating its effectiveness in terms of data efficiency. Future research will focus on extending the Multitask learning framework to encompass additional recognition tasks.
ContentCTR: Frame-level Live Streaming Click-Through Rate Prediction with Multimodal Transformer
Venue: arXiv cs.CV 2023
Year: 2023

In recent years, live streaming platforms have gained immense popularity as they allow users to broadcast their videos and interact in real-time with hosts and peers. Due to the dynamic changes of live content, accurate recommendation models are crucial for enhancing user experience. However, most previous works treat the live as a whole item and explore the Click-through-Rate (CTR) prediction framework on item-level, neglecting that the dynamic changes that occur even within the same live room. In this paper, we proposed a ContentCTR model that leverages multimodal transformer for frame-level CTR prediction. First, we present an end-to-end framework that can make full use of multimodal information, including visual frames, audio, and comments, to identify the most attractive live frames. Second, to prevent the model from collapsing into a mediocre solution, a novel pairwise loss function with first-order difference constraints is proposed to utilize the contrastive information existing in the highlight and non-highlight frames. Additionally, we design a temporal text-video alignment module based on Dynamic Time Warping to eliminate noise caused by the ambiguity and non-sequential alignment of visual and textual information. We conduct extensive experiments on both real-world scenarios and public datasets, and our ContentCTR model outperforms traditional recommendation models in capturing real-time content changes. Moreover, we deploy the proposed method on our company platform, and the results of online A/B testing further validate its practical significance.
Contrastive Multi-view Framework for Customer Lifetime Value Prediction
Venue: arXiv cs.IR 2023
Year: 2023

Accurate customer lifetime value (LTV) prediction can help service providers optimize their marketing policies in customer-centric applications. However, the heavy sparsity of consumption events and the interference of data variance and noise obstruct LTV estimation. Many existing LTV prediction methods directly train a single-view LTV predictor on consumption samples, which may yield inaccurate and even biased knowledge extraction. In this paper, we propose a contrastive multi-view framework for LTV prediction, which is a plug-and-play solution compatible with various backbone models. It synthesizes multiple heterogeneous LTV regressors with complementary knowledge to improve model robustness and captures sample relatedness via contrastive learning to mitigate the dependency on data abundance. Concretely, we use a decomposed scheme that converts the LTV prediction problem into a combination of estimating consumption probability and payment amount. To alleviate the impact of noisy data on model learning, we propose a multi-view framework that jointly optimizes multiple types of regressors with diverse characteristics and advantages to encode and fuse comprehensive knowledge. To fully exploit the potential of limited training samples, we propose a hybrid contrastive learning method to help capture the relatedness between samples in both classification and regression tasks. We conduct extensive experiments on a real-world game LTV prediction dataset and the results validate the effectiveness of our method. We have deployed our solution online in Huawei's mobile game center and achieved 32.26% of total payment amount gains.
Cloud-Native Computing: A Survey from the Perspective of Services
Venue: arXiv cs.DC 2023
Year: 2023

The development of cloud computing delivery models inspires the emergence of cloud-native computing. Cloud-native computing, as the most influential development principle for web applications, has already attracted increasingly more attention in both industry and academia. Despite the momentum in the cloud-native industrial community, a clear research roadmap on this topic is still missing. As a contribution to this knowledge, this paper surveys key issues during the life-cycle of cloud-native applications, from the perspective of services. Specifically, we elaborate the research domains by decoupling the life-cycle of cloud-native applications into four states: building, orchestration, operate, and maintenance. We also discuss the fundamental necessities and summarize the key performance metrics that play critical roles during the development and management of cloud-native applications. We highlight the key implications and limitations of existing works in each state. The challenges, future directions, and research opportunities are also discussed.
Homomorphic Encryption: An Analysis of its Applications in Searchable Encryption
Venue: arXiv cs.CR 2023
Year: 2023

The widespread adoption of cloud infrastructures has revolutionised data storage and access. However, it has also raised concerns regarding the privacy of sensitive data stored in the cloud. To address these concerns, encryption techniques have been widely used. However, traditional encryption schemes limit the efficient search and retrieval of encrypted data. To tackle this challenge, innovative approaches have emerged, such as the utilisation of Homomorphic Encryption (HE) in Searchable Encryption (SE) schemes. This paper provides a comprehensive analysis of the advancements in HE-based privacy-preserving techniques, focusing on their application in SE. The main contributions of this work include the identification and classification of existing SE schemes that utilize HE, a comprehensive analysis of the types of HE used in SE, an examination of how HE shapes the search process structure and enables additional functionalities, and the identification of promising directions for future research in HE-based SE. The findings reveal the increasing usage of HE in SE schemes, particularly Partially Homomorphic Encryption. The analysis also highlights the prevalence of index-based SE schemes using HE, the support for ranked search and multi-keyword queries, and the need for further exploration in functionalities such as verifiability and the ability to authorise and revoke users. Future research directions include exploring the usage of other encryption schemes alongside HE, addressing omissions in functionalities like fuzzy keyword search, and leveraging recent advancements in Fully Homomorphic Encryption schemes.
Federated Learning on Non-iid Data via Local and Global Distillation
Venue: arXiv cs.LG 2023
Year: 2023

Most existing federated learning algorithms are based on the vanilla FedAvg scheme. However, with the increase of data complexity and the number of model parameters, the amount of communication traffic and the number of iteration rounds for training such algorithms increases significantly, especially in non-independently and homogeneously distributed scenarios, where they do not achieve satisfactory performance. In this work, we propose FedND: federated learning with noise distillation. The main idea is to use knowledge distillation to optimize the model training process. In the client, we propose a self-distillation method to train the local model. In the server, we generate noisy samples for each client and use them to distill other clients. Finally, the global model is obtained by the aggregation of local models. Experimental results show that the algorithm achieves the best performance and is more communication-efficient than state-of-the-art methods.
Hierarchical Matching and Reasoning for Multi-Query Image Retrieval
Venue: arXiv cs.CV 2023
Year: 2023

As a promising field, Multi-Query Image Retrieval (MQIR) aims at searching for the semantically relevant image given multiple region-specific text queries. Existing works mainly focus on a single-level similarity between image regions and text queries, which neglects the hierarchical guidance of multi-level similarities and results in incomplete alignments. Besides, the high-level semantic correlations that intrinsically connect different region-query pairs are rarely considered. To address above limitations, we propose a novel Hierarchical Matching and Reasoning Network (HMRN) for MQIR. It disentangles MQIR into three hierarchical semantic representations, which is responsible to capture fine-grained local details, contextual global scopes, and high-level inherent correlations. HMRN comprises two modules: Scalar-based Matching (SM) module and Vector-based Reasoning (VR) module. Specifically, the SM module characterizes the multi-level alignment similarity, which consists of a fine-grained local-level similarity and a context-aware global-level similarity. Afterwards, the VR module is developed to excavate the potential semantic correlations among multiple region-query pairs, which further explores the high-level reasoning similarity. Finally, these three-level similarities are aggregated into a joint similarity space to form the ultimate similarity. Extensive experiments on the benchmark dataset demonstrate that our HMRN substantially surpasses the current state-of-the-art methods. For instance, compared with the existing best method Drill-down, the metric R@1 in the last round is improved by 23.4%. Our source codes will be released at https://github.com/LZH-053/HMRN.
Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation
Venue: arXiv cs.IR 2023
Year: 2023

Recommendation systems suffer in the strict cold-start (SCS) scenario, where the user-item interactions are entirely unavailable. The ID-based approaches completely fail to work. Cold-start recommenders, on the other hand, leverage item contents to map the new items to the existing ones. However, the existing SCS recommenders explore item contents in coarse-grained manners that introduce noise or information loss. Moreover, informative data sources other than item contents, such as users' purchase sequences and review texts, are ignored. We explore the role of the fine-grained item attributes in bridging the gaps between the existing and the SCS items and pre-train a knowledgeable item-attribute graph for SCS item recommendation. Our proposed framework, ColdGPT, models item-attribute correlations into an item-attribute graph by extracting fine-grained attributes from item contents. ColdGPT then transfers knowledge into the item-attribute graph from various available data sources, i.e., item contents, historical purchase sequences, and review texts of the existing items, via multi-task learning. To facilitate the positive transfer, ColdGPT designs submodules according to the natural forms of the data sources and coordinates the multiple pre-training tasks via unified alignment-and-uniformity losses. Our pre-trained item-attribute graph acts as an implicit, extendable item embedding matrix, which enables the SCS item embeddings to be easily acquired by inserting these items and propagating their attributes' embeddings. We carefully process three public datasets, i.e., Yelp, Amazon-home, and Amazon-sports, to guarantee the SCS setting for evaluation. Extensive experiments show that ColdGPT consistently outperforms the existing SCS recommenders by large margins and even surpasses models that are pre-trained on 75-224 times more, cross-domain data on two out of four datasets.