bluuebunny/crossref_metadata_2025 huggingface dataset:
Correspondence-Free Domain Alignment for Unsupervised Cross-Domain Image Retrieval
Authors: Wang Xu , Peng Dezhong , Yan Ming , Hu Peng
Published: June 2023

Cross-domain image retrieval aims at retrieving images across different domains to excavate cross-domain classificatory or correspondence relationships. This paper studies a less-touched problem of cross-domain image retrieval, i.e., unsupervised cross-domain image retrieval, considering the following practical assumptions: (i) no correspondence relationship, and (ii) no category annotations. It is challenging to align and bridge distinct domains without cross-domain correspondence. To tackle the challenge, we present a novel Correspondence-free Domain Alignment (CoDA) method to effectively eliminate the cross-domain gap through In-domain Self-matching Supervision (ISS) and Cross-domain Classifier Alignment (CCA). To be specific, ISS is presented to encapsulate discriminative information into the latent common space by elaborating a novel self-matching supervision mechanism. To alleviate the cross-domain discrepancy, CCA is proposed to align distinct domain-specific classifiers. Thanks to the ISS and CCA, our method could encode the discrimination into the domain-invariant embedding space for unsupervised cross-domain image retrieval. To verify the effectiveness of the proposed method, extensive experiments are conducted on four benchmark datasets compared with six state-of-the-art methods.
SEnsor Alignment for Multivariate Time-Series Unsupervised Domain Adaptation
Authors: Wang Yucheng , Xu Yuecong , Yang Jianfei , Chen Zhenghua , Wu Min , Li Xiaoli , Xie Lihua
Published: June 2023

Unsupervised Domain Adaptation (UDA) methods can reduce label dependency by mitigating the feature discrepancy between labeled samples in a source domain and unlabeled samples in a similar yet shifted target domain. Though achieving good performance, these methods are inapplicable for Multivariate Time-Series (MTS) data. MTS data are collected from multiple sensors, each of which follows various distributions. However, most UDA methods solely focus on aligning global features but cannot consider the distinct distributions of each sensor. To cope with such concerns, a practical domain adaptation scenario is formulated as Multivariate Time-Series Unsupervised Domain Adaptation (MTS-UDA). In this paper, we propose SEnsor Alignment (SEA) for MTS-UDA to reduce the domain discrepancy at both the local and global sensor levels. At the local sensor level, we design the endo-feature alignment to align sensor features and their correlations across domains, whose information represents the features of each sensor and the interactions between sensors. Further, to reduce domain discrepancy at the global sensor level, we design the exo-feature alignment to enforce restrictions on the global sensor features. Meanwhile, MTS also incorporates the essential spatial-temporal dependencies information between sensors, which cannot be transferred by existing UDA methods. Therefore, we model the spatial-temporal information of MTS with a multi-branch self-attention mechanism for simple and effective transfer across domains. Empirical results demonstrate the state-of-the-art performance of our proposed SEA on two public MTS datasets for MTS-UDA. The code is available at
https://github.com/Frank-Wang-oss/SEA
Unlabeled Imperfect Demonstrations in Adversarial Imitation Learning
Authors: Wang Yunke , Du Bo , Xu Chang
Published: June 2023

Adversarial imitation learning has become a widely used imitation learning framework. The discriminator is often trained by taking expert demonstrations and policy trajectories as examples respectively from two categories (positive vs. negative) and the policy is then expected to produce trajectories that are indistinguishable from the expert demonstrations. But in the real world, the collected expert demonstrations are more likely to be imperfect, where only an unknown fraction of the demonstrations are optimal. Instead of treating imperfect expert demonstrations as absolutely positive or negative, we investigate unlabeled imperfect expert demonstrations as they are. A positive-unlabeled adversarial imitation learning algorithm is developed to dynamically sample expert demonstrations that can well match the trajectories from the constantly optimized agent policy. The trajectories of an initial agent policy could be closer to those non-optimal expert demonstrations, but within the framework of adversarial imitation learning, agent policy will be optimized to cheat the discriminator and produce trajectories that are similar to those optimal expert demonstrations. Theoretical analysis shows that our method learns from the imperfect demonstrations via a self-paced way. Experimental results on MuJoCo and RoboSuite platforms demonstrate the effectiveness of our method from different aspects.
Faster Adaptive Federated Learning
Authors: Wu Xidong , Huang Feihu , Hu Zhengmian , Huang Heng
Published: June 2023

Federated learning has attracted increasing attention with the emergence of distributed data. While extensive federated learning algorithms have been proposed for the non-convex distributed problem, the federated learning in practice still faces numerous challenges, such as the large training iterations to converge since the sizes of models and datasets keep increasing, and the lack of adaptivity by SGD-based model updates. Meanwhile, the study of adaptive methods in federated learning is scarce and existing works either lack a complete theoretical convergence guarantee or have slow sample complexity. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on the momentum-based variance reduced technique in cross-silo FL. We first explore how to design the adaptive algorithm in the FL setting. By providing a counter-example, we prove that a simple combination of FL and adaptive methods could lead to divergence. More importantly, we provide a convergence analysis for our method and prove that our algorithm is the first adaptive FL algorithm to reach the best-known samples O(epsilon(-3)) and O(epsilon(-2)) communication rounds to find an epsilon-stationary point without large batches. The experimental results on the language modeling task and image classification task with heterogeneous data demonstrate the efficiency of our algorithms.
Differentially Private Learning with Per-Sample Adaptive Clipping
Authors: Xia Tianyu , Shen Shuheng , Yao Su , Fu Xinyi , Xu Ke , Xu Xiaolong , Fu Xing
Published: June 2023

Privacy in AI remains a topic that draws attention from researchers and the general public in recent years. As one way to implement privacy-preserving AI, differentially private learning is a framework that enables AI models to use differential privacy (DP). To achieve DP in the learning process, existing algorithms typically limit the magnitude of gradients with a constant clipping, which requires carefully tuned due to its significant impact on model performance. As a solution to this issue, latest works NSGD and Auto-S innovatively propose to use normalization instead of clipping to avoid hyperparameter tuning. However, normalization-based approaches like NSGD and Auto-S rely on a monotonic weight function, which imposes excessive weight on small gradient samples and introduces extra deviation to the update. In this paper, we propose a Differentially Private Per-Sample Adaptive Clipping (DP-PSAC) algorithm based on a non-monotonic adaptive weight function, which guarantees privacy without the typical hyperparameter tuning process of using a constant clipping while significantly reducing the deviation between the update and true batch-averaged gradient. We provide a rigorous theoretical convergence analysis and show that with convergence rate at the same order, the proposed algorithm achieves a lower non-vanishing bound, which is maintained over training iterations, compared with NSGD/Auto-S. In addition, through extensive experimental evaluation, we show that DP-PSAC outperforms or matches the state-of-the-art methods on multiple main-stream vision and language tasks.
Efficient Top-K Feature Selection Using Coordinate Descent Method
Authors: Xu Lei , Wang Rong , Nie Feiping , Li Xuelong
Published: June 2023

Sparse learning based feature selection has been widely investigated in recent years. In this study, we focus on the l2,0-norm based feature selection, which is effective for exact top-k feature selection but challenging to optimize. To solve the general l2,0-norm constrained problems, we novelly develop a parameter-free optimization framework based on the coordinate descend (CD) method, termed CD-LSR. Specifically, we devise a skillful conversion from the original problem to solving one continuous matrix and one discrete selection matrix. Then the nontrivial l2,0-norm constraint can be solved efficiently by solving the selection matrix with CD method. We impose the l2,0-norm on a vanilla least square regression (LSR) model for feature selection and optimize it with CD-LSR. Extensive experiments exhibit the efficiency of CD-LSR, as well as the discrimination ability of l2,0-norm to identify informative features. More importantly, the versatility of CD-LSR facilitates the applications of the l2,0-norm in more sophisticated models. Based on the competitive performance of l2,0-norm on the baseline LSR model, the satisfactory performance of its applications is reasonably expected. The source MATLAB code are available at: https://github.com/solerxl/Code_For_AAAI_2023.
Global Concept-Based Interpretability for Graph Neural Networks via Neuron Analysis
Authors: Xuanyuan Han , Barbiero Pietro , Georgiev Dobrik , Magister Lucie Charlotte , Liò Pietro
Published: June 2023

Graph neural networks (GNNs) are highly effective on a variety of graph-related tasks; however, they lack interpretability and transparency. Current explainability approaches are typically local and treat GNNs as black-boxes. They do not look inside the model, inhibiting human trust in the model and explanations. Motivated by the ability of neurons to detect high-level semantic concepts in vision models, we perform a novel analysis on the behaviour of individual GNN neurons to answer questions about GNN interpretability. We propose a novel approach for producing global explanations for GNNs using neuron-level concepts to enable practitioners to have a high-level view of the model. Specifically, (i) to the best of our knowledge, this is the first work which shows that GNN neurons act as concept detectors and have strong alignment with concepts formulated as logical compositions of node degree and neighbourhood properties; (ii) we quantitatively assess the importance of detected concepts, and identify a trade-off between training duration and neuron-level interpretability; (iii) we demonstrate that our global explainability approach has advantages over the current state-of-the-art -- we can disentangle the explanation into individual interpretable concepts backed by logical descriptions, which reduces potential for bias and improves user-friendliness.
T2G-FORMER: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interaction
Authors: Yan Jiahuan , Chen Jintai , Wu Yixuan , Chen Danny Z. , Wu Jian
Published: June 2023

Recent development of deep neural networks (DNNs) for tabular learning has largely benefited from the capability of DNNs for automatic feature interaction. However, the heterogeneity nature of tabular features makes such features relatively independent, and developing effective methods to promote tabular feature interaction still remains an open problem. In this paper, we propose a novel Graph Estimator, which automatically estimates the relations among tabular features and builds graphs by assigning edges between related features. Such relation graphs organize independent tabular features into a kind of graph data such that interaction of nodes (tabular features) can be conducted in an orderly fashion. Based on our proposed Graph Estimator, we present a bespoke Transformer network tailored for tabular learning, called T2G-Former, which processes tabular data by performing tabular feature interaction guided by the relation graphs. A specific Cross-level Readout collects salient features predicted by the layers in T2G-Former across different levels, and attains global semantics for final prediction. Comprehensive experiments show that our T2G-Former achieves superior performance among DNNs and is competitive with non-deep Gradient Boosted Decision Tree models. The code and detailed results are available at https://github.com/jyansir/t2g-former.
Reinforcement Causal Structure Learning on Order Graph
Authors: Yang Dezhi , Yu Guoxian , Wang Jun , Wu Zhengtian , Guo Maozu
Published: June 2023

Learning directed acyclic graph (DAG) that describes the causality of observed data is a very challenging but important task. Due to the limited quantity and quality of observed data, and non-identifiability of causal graph, it is almost impossible to infer a single precise DAG. Some methods approximate the posterior distribution of DAGs to explore the DAG space via Markov chain Monte Carlo (MCMC), but the DAG space is over the nature of super-exponential growth, accurately characterizing the whole distribution over DAGs is very intractable. In this paper, we propose Reinforcement Causal Structure Learning on Order Graph (RCL-OG) that uses order graph instead of MCMC to model different DAG topological orderings and to reduce the problem size. RCL-OG first defines reinforcement learning with a new reward mechanism to approximate the posterior distribution of orderings in an efficacy way, and uses deep Q-learning to update and transfer rewards between nodes. Next, it obtains the probability transition model of nodes on order graph, and computes the posterior probability of different orderings. In this way, we can sample on this model to obtain the ordering with high probability. Experiments on synthetic and benchmark datasets show that RCL-OG provides accurate posterior probability approximation and achieves better results than competitive causal discovery algorithms.
Certifiable Out-of-Distribution Generalization
Authors: Ye Nanyang , Zhu Lin , Wang Jia , Zeng Zhaoyu , Shao Jiayao , Peng Chensheng , Pan Bikang , Li Kaican , Zhu Jun
Published: June 2023

Machine learning methods suffer from test-time performance degeneration when faced with out-of-distribution (OoD) data whose distribution is not necessarily the same as training data distribution. Although a plethora of algorithms have been proposed to mitigate this issue, it has been demonstrated that achieving better performance than ERM simultaneously on different types of distributional shift datasets is challenging for existing approaches. Besides, it is unknown how and to what extent these methods work on any OoD datum without theoretical guarantees. In this paper, we propose a certifiable out-of-distribution generalization method that provides provable OoD generalization performance guarantees via a functional optimization framework leveraging random distributions and max-margin learning for each input datum. With this approach, the proposed algorithmic scheme can provide certified accuracy for each input datum's prediction on the semantic space and achieves better performance simultaneously on OoD datasets dominated by correlation shifts or diversity shifts. Our code is available at https://github.com/ZlatanWilliams/StochasticDisturbanceLearning.
Coordinate Descent Methods for DC Minimization: Optimality Conditions and Global Convergence
Authors: Yuan Ganzhao
Published: June 2023

Difference-of-Convex (DC) minimization, referring to the problem of minimizing the difference of two convex functions, has been found rich applications in statistical learning and studied extensively for decades. However, existing methods are primarily based on multi-stage convex relaxation, only leading to weak optimality of critical points. This paper proposes a coordinate descent method for minimizing a class of DC functions based on sequential nonconvex approximation. Our approach iteratively solves a nonconvex one-dimensional subproblem globally, and it is guaranteed to converge to a coordinate-wise stationary point. We prove that this new optimality condition is always stronger than the standard critical point condition and directional point condition under a mildlocally bounded nonconvexity assumption. For comparisons, we also include a naive variant of coordinate descent methods based on sequential convex approximation in our study. When the objective function satisfies a globally bounded nonconvexity assumption and Luo-Tseng error bound assumption, coordinate descent methods achieve Q-linear convergence rate. Also, for many applications of interest, we show that the nonconvex one-dimensional subproblem can be computed exactly and efficiently using a breakpoint searching method. Finally, we have conducted extensive experiments on several statistical learning tasks to show the superiority of our approach.
Value-Consistent Representation Learning for Data-Efficient Reinforcement Learning
Authors: Yue Yang , Kang Bingyi , Xu Zhongwen , Huang Gao , Yan Shuicheng
Published: June 2023

Deep reinforcement learning (RL) algorithms suffer severe performance degradation when the interaction data is scarce, which limits their real-world application. Recently, visual representation learning has been shown to be effective and promising for boosting sample efficiency in RL. These methods usually rely on contrastive learning and data augmentation to train a transition model, which is different from how the model is used in RL---performing value-based planning. Accordingly, the learned representation by these visual methods may be good for recognition but not optimal for estimating state value and solving the decision problem. To address this issue, we propose a novel method, called value-consistent representation learning (VCR), to learn representations that are directly related to decision-making. More specifically, VCR trains a model to predict the future state (also referred to as the "imagined state'') based on the current one and a sequence of actions. Instead of aligning this imagined state with a real state returned by the environment, VCR applies a Q value head on both of the states and obtains two distributions of action values. Then a distance is computed and minimized to force the imagined state to produce a similar action value prediction as that by the real state. We develop two implementations of the above idea for the discrete and continuous action spaces respectively. We conduct experiments on Atari 100k and DeepMind Control Suite benchmarks to validate their effectiveness for improving sample efficiency. It has been demonstrated that our methods achieve new state-of-the-art performance for search-free RL algorithms.
Learning Conflict-Noticed Architecture for Multi-Task Learning
Authors: Yue Zhixiong , Zhang Yu , Liang Jie
Published: June 2023

Multi-task learning has been widely used in many applications to enable more efficient learning by sharing part of the architecture across multiple tasks. However, a major challenge is the gradient conflict when optimizing the shared parameters, where the gradients of different tasks could have opposite directions. Directly averaging those gradients will impair the performance of some tasks and cause negative transfer. Different from most existing works that manipulate gradients to mitigate the gradient conflict, in this paper, we address this problem from the perspective of architecture learning and propose a Conflict-Noticed Architecture Learning (CoNAL) method to alleviate the gradient conflict by learning architectures. By introducing purely-specific modules specific to each task in the search space, the CoNAL method can automatically learn when to switch to purely-specific modules in the tree-structured network architectures when the gradient conflict occurs. To handle multi-task problems with a large number of tasks, we propose a progressive extension of the CoNAL method. Extensive experiments on computer vision, natural language processing, and reinforcement learning benchmarks demonstrate the effectiveness of the proposed methods.
Foresee What You Will Learn: Data Augmentation for Domain Generalization in Non-stationary Environment
Authors: Zeng Qiuhao , Wang Wei , Zhou Fan , Ling Charles , Wang Boyu
Published: June 2023

Existing domain generalization aims to learn a generalizable model to perform well even on unseen domains. For many real-world machine learning applications, the data distribution often shifts gradually along domain indices. For example, a self-driving car with a vision system drives from dawn to dusk, with the sky gradually darkening. Therefore, the system must be able to adapt to changes in ambient illuminations and continue to drive safely on the road. In this paper, we formulate such problems as Evolving Domain Generalization, where a model aims to generalize well on a target domain by discovering and leveraging the evolving pattern of the environment. We then propose Directional Domain Augmentation (DDA), which simulates the unseen target features by mapping source data as augmentations through a domain transformer. Specifically, we formulate DDA as a bi-level optimization problem and solve it through a novel meta-learning approach in the representation space. We evaluate the proposed method on both synthetic datasets and real-world datasets, and empirical results show that our approach can outperform other existing methods.
Rethinking Alignment and Uniformity in Unsupervised Image Semantic Segmentation
Authors: Zhang Daoan , Li Chenming , Li Haoquan , Huang Wenjian , Huang Lingyun , Zhang Jianguo
Published: June 2023

Unsupervised image segmentation aims to match low-level visual features with semantic-level representations without outer supervision. In this paper, we address the critical properties from the view of feature alignments and feature uniformity for UISS models. We also make a comparison between UISS and image-wise representation learning. Based on the analysis, we argue that the existing MI-based methods in UISS suffer from representation collapse. By this, we proposed a robust network called Semantic Attention Network(SAN), in which a new module Semantic Attention(SEAT) is proposed to generate pixel-wise and semantic features dynamically. Experimental results on multiple semantic segmentation benchmarks show that our unsupervised segmentation framework specializes in catching semantic representations, which outperforms all the unpretrained and even several pretrained methods.
When Neural Networks Fail to Generalize? A Model Sensitivity Perspective
Authors: Zhang Jiajin , Chao Hanqing , Dhurandhar Amit , Chen Pin-Yu , Tajer Ali , Xu Yangyang , Yan Pingkun
Published: June 2023

Domain generalization (DG) aims to train a model to perform well in unseen domains under different distributions. This paper considers a more realistic yet more challenging scenario, namely Single Domain Generalization (Single-DG), where only a single source domain is available for training. To tackle this challenge, we first try to understand when neural networks fail to generalize? We empirically ascertain a property of a model that correlates strongly with its generalization that we coin as "model sensitivity". Based on our analysis, we propose a novel strategy of Spectral Adversarial Data Augmentation (SADA) to generate augmented images targeted at the highly sensitive frequencies. Models trained with these hard-to-learn samples can effectively suppress the sensitivity in the frequency space, which leads to improved generalization performance. Extensive experiments on multiple public datasets demonstrate the superiority of our approach, which surpasses the state-of-the-art single-DG methods by up to 2.55%. The source code is available at https://github.com/DIAL-RPI/Spectral-Adversarial-Data-Augmentation.
FedALA: Adaptive Local Aggregation for Personalized Federated Learning
Authors: Zhang Jianqing , Hua Yang , Wang Hao , Song Tao , Xue Zhengui , Ma Ruhui , Guan Haibing
Published: June 2023

A key challenge in federated learning (FL) is the statistical heterogeneity that impairs the generalization of the global model on each client. To address this, we propose a method Federated learning with Adaptive Local Aggregation (FedALA) by capturing the desired information in the global model for client models in personalized FL. The key component of FedALA is an Adaptive Local Aggregation (ALA) module, which can adaptively aggregate the downloaded global model and local model towards the local objective on each client to initialize the local model before training in each iteration. To evaluate the effectiveness of FedALA, we conduct extensive experiments with five benchmark datasets in computer vision and natural language processing domains. FedALA outperforms eleven state-of-the-art baselines by up to 3.27% in test accuracy. Furthermore, we also apply ALA module to other federated learning methods and achieve up to 24.19% improvement in test accuracy. Code is available at https://github.com/TsingZ0/FedALA.
Spectral Feature Augmentation for Graph Contrastive Learning and Beyond
Authors: Zhang Yifei , Zhu Hao , Song Zixing , Koniusz Piotr , King Irwin
Published: June 2023

Although augmentations (e.g., perturbation of graph edges, image crops) boost the efficiency of Contrastive Learning (CL), feature level augmentation is another plausible, complementary yet not well researched strategy. Thus, we present a novel spectral feature argumentation for contrastive learning on graphs (and images). To this end, for each data view, we estimate a low-rank approximation per feature map and subtract that approximation from the map to obtain its complement. This is achieved by the proposed herein incomplete power iteration, a non-standard power iteration regime which enjoys two valuable byproducts (under mere one or two iterations): (i) it partially balances spectrum of the feature map, and (ii) it injects the noise into rebalanced singular values of the feature map (spectral augmentation). For two views, we align these rebalanced feature maps as such an improved alignment step can focus more on less dominant singular values of matrices of both views, whereas the spectral augmentation does not affect the spectral angle alignment (singular vectors are not perturbed). We derive the analytical form for: (i) the incomplete power iteration to capture its spectrum-balancing effect, and (ii) the variance of singular values augmented implicitly by the noise. We also show that the spectral augmentation improves the generalization bound. Experiments on graph/image datasets show that our spectral feature augmentation outperforms baselines, and is complementary with other augmentation strategies and compatible with various contrastive losses.
Dynamic Heterogeneous Graph Attention Neural Architecture Search
Authors: Zhang Zeyang , Zhang Ziwei , Wang Xin , Qin Yijian , Qin Zhou , Zhu Wenwu
Published: June 2023

Dynamic heterogeneous graph neural networks (DHGNNs) have been shown to be effective in handling the ubiquitous dynamic heterogeneous graphs. However, the existing DHGNNs are hand-designed, requiring extensive human efforts and failing to adapt to diverse dynamic heterogeneous graph scenarios. In this paper, we propose to automate the design of DHGNN, which faces two major challenges: 1) how to design the search space to jointly consider the spatial-temporal dependencies and heterogeneous interactions in graphs; 2) how to design an efficient search algorithm in the potentially large and complex search space. To tackle these challenges, we propose a novel Dynamic Heterogeneous Graph Attention Search (DHGAS) method. Our proposed method can automatically discover the optimal DHGNN architecture and adapt to various dynamic heterogeneous graph scenarios without human guidance. In particular, we first propose a unified dynamic heterogeneous graph attention (DHGA) framework, which enables each node to jointly attend its heterogeneous and dynamic neighbors. Based on the framework, we design a localization space to determine where the attention should be applied and a parameterization space to determine how the attention should be parameterized. Lastly, we design a multi-stage differentiable search algorithm to efficiently explore the search space. Extensive experiments on real-world dynamic heterogeneous graph datasets demonstrate that our proposed method significantly outperforms state-of-the-art baselines for tasks including link prediction, node classification and node regression. To the best of our knowledge, DHGAS is the first dynamic heterogeneous graph neural architecture search method.
Fairness and Explainability: Bridging the Gap towards Fair Model Explanations
Authors: Zhao Yuying , Wang Yu , Derr Tyler
Published: June 2023

While machine learning models have achieved unprecedented success in real-world applications, they might make biased/unfair decisions for specific demographic groups and hence result in discriminative outcomes. Although research efforts have been devoted to measuring and mitigating bias, they mainly study bias from the result-oriented perspective while neglecting the bias encoded in the decision-making procedure. This results in their inability to capture procedure-oriented bias, which therefore limits the ability to have a fully debiasing method. Fortunately, with the rapid development of explainable machine learning, explanations for predictions are now available to gain insights into the procedure. In this work, we bridge the gap between fairness and explainability by presenting a novel perspective of procedure-oriented fairness based on explanations. We identify the procedure-based bias by measuring the gap of explanation quality between different groups with Ratio-based and Value-based Explanation Fairness. The new metrics further motivate us to design an optimization objective to mitigate the procedure-based bias where we observe that it will also mitigate bias from the prediction. Based on our designed optimization objective, we propose a Comprehensive Fairness Algorithm (CFA), which simultaneously fulfills multiple objectives - improving traditional fairness, satisfying explanation fairness, and maintaining the utility performance. Extensive experiments on real-world datasets demonstrate the effectiveness of our proposed CFA and highlight the importance of considering fairness from the explainability perspective. Our code: https://github.com/YuyingZhao/FairExplanations-CFA.
Adaptive Policy Learning for Offline-to-Online Reinforcement Learning
Authors: Zheng Han , Luo Xufang , Wei Pengfei , Song Xuan , Li Dongsheng , Jiang Jing
Published: June 2023

Conventional reinforcement learning (RL) needs an environment to collect fresh data, which is impractical when online interactions are costly. Offline RL provides an alternative solution by directly learning from the previously collected dataset. However, it will yield unsatisfactory performance if the quality of the offline datasets is poor. In this paper, we consider an offline-to-online setting where the agent is first learned from the offline dataset and then trained online, and propose a framework called Adaptive Policy Learning for effectively taking advantage of offline and online data. Specifically, we explicitly consider the difference between the online and offline data and apply an adaptive update scheme accordingly, that is, a pessimistic update strategy for the offline dataset and an optimistic/greedy update scheme for the online dataset. Such a simple and effective method provides a way to mix the offline and online RL and achieve the best of both worlds. We further provide two detailed algorithms for implementing the framework through embedding value or policy-based RL algorithms into it. Finally, we conduct extensive experiments on popular continuous control tasks, and results show that our algorithm can learn the expert policy with high sample efficiency even when the quality of offline dataset is poor, e.g., random dataset.
CowClip: Reducing CTR Prediction Model Training Time from 12 Hours to 10 Minutes on 1 GPU
Authors: Zheng Zangwei , Xu Pengtai , Zou Xuan , Tang Da , Li Zhen , Xi Chenguang , Wu Peng , Zou Leqi , Zhu Yijie , Chen Ming , Ding Xiangzhuo , Xue Fuzhao , Qin Ziheng , Cheng Youlong , You Yang
Published: June 2023

The click-through rate (CTR) prediction task is to predict whether a user will click on the recommended item. As mind-boggling amounts of data are produced online daily, accelerating CTR prediction model training is critical to ensuring an up-to-date model and reducing the training cost. One approach to increase the training speed is to apply large batch training. However, as shown in computer vision and natural language processing tasks, training with a large batch easily suffers from the loss of accuracy. Our experiments show that previous scaling rules fail in the training of CTR prediction neural networks. To tackle this problem, we first theoretically show that different frequencies of ids make it challenging to scale hyperparameters when scaling the batch size. To stabilize the training process in a large batch size setting, we develop the adaptive Column-wise Clipping (CowClip). It enables an easy and effective scaling rule for the embeddings, which keeps the learning rate unchanged and scales the L2 loss. We conduct extensive experiments with four CTR prediction networks on two real-world datasets and successfully scaled 128 times the original batch size without accuracy loss. In particular, for CTR prediction model DeepFM training on the Criteo dataset, our optimization framework enlarges the batch size from 1K to 128K with over 0.1% AUC improvement and reduces training time from 12 hours to 10 minutes on a single V100 GPU. Our code locates at github.com/bytedance/LargeBatchCTR.
SLOTH: Structured Learning and Task-Based Optimization for Time Series Forecasting on Hierarchies
Authors: Zhou Fan , Pan Chen , Ma Lintao , Liu Yu , Wang Shiyu , Zhang James , Zhu Xinxin , Hu Xuanwei , Hu Yunhua , Zheng Yangfei , Lei Lei , Yun Hu
Published: June 2023

Multivariate time series forecasting with hierarchical structure
is widely used in real-world applications, e.g., sales
predictions for the geographical hierarchy formed by cities,
states, and countries. The hierarchical time series (HTS) forecasting
includes two sub-tasks, i.e., forecasting and reconciliation.
In the previous works, hierarchical information is only
integrated in the reconciliation step to maintain coherency,
but not in forecasting step for accuracy improvement. In this
paper, we propose two novel tree-based feature integration
mechanisms, i.e., top-down convolution and bottom-up attention
to leverage the information of the hierarchical structure
to improve the forecasting performance. Moreover, unlike
most previous reconciliation methods which either rely
on strong assumptions or focus on coherent constraints only,
we utilize deep neural optimization networks, which not only
achieve coherency without any assumptions, but also allow
more flexible and realistic constraints to achieve task-based
targets, e.g., lower under-estimation penalty and meaningful
decision-making loss to facilitate the subsequent downstream
tasks. Experiments on real-world datasets demonstrate that
our tree-based feature integration mechanism achieves superior
performances on hierarchical forecasting tasks compared
to the state-of-the-art methods, and our neural optimization
networks can be applied to real-world tasks effectively without
any additional effort under coherence and task-based constraints.
Robust Temporal Smoothness in Multi-Task Learning
Authors: Zhou Menghui , Zhang Yu , Yang Yun , Liu Tong , Yang Po
Published: June 2023

Multi-task learning models based on temporal smoothness assumption, in which each time point of a sequence of time points concerns a task of prediction, assume the adjacent tasks are similar to each other. However, the effect of outliers is not taken into account. In this paper, we show that even only one outlier task will destroy the performance of the entire model. To solve this problem, we propose two Robust Temporal Smoothness (RoTS) frameworks. Compared with the existing models based on temporal relation, our methods not only chase the temporal smoothness information but identify outlier tasks, however, without increasing the computational complexity. Detailed theoretical analyses are presented to evaluate the performance of our methods. Experimental results on synthetic and real-life datasets demonstrate the effectiveness of our frameworks. We also discuss several potential specific applications and extensions of our RoTS frameworks.
THREE-DIMENSIONAL NUMERICAL ANALYSIS OF FLOW AND HEAT TRANSFER CHARACTERISTICS OF OIL/GAS TWO-PHASE MIXTURE IN AERO-ENGINE BEARING CHAMBERS OF DIFFERENT STRUCTURES
Authors: Wang Wei , Li Caixia , Wan Pan , Lata Bhusal Manju , Li Shaobai
Published: June 2023

The change of the outlet number of bearing chambers can change the pressure distribution in the chamber and the
action of the air shear force, which has an important influence on oil/gas two-phase flow and heat transfer. In this paper, the flow and heat transfer characteristics in the ventless bearing chamber and the vented bearing chamber were compared and analyzed by numerical simulation. The accuracy of the numerical model was verified by comparison
with the experimental data. Three different rotor speeds were selected, and the lubricating oil distribution, the velocity of oil/gas two-phase flow, and the temperature distribution in the bearing chamber were investigated under different numbers of outlets of the bearing chamber. The heat flux and Nusselt number distribution of convective heat transfer between oil/gas two-phase flow and the outer wall at different rotor speeds were analyzed. The results show that the thickness of oil film increases in the ventless bearing chamber with increase of rotor speed, and when air flow is large, it hinders the uniformity of lubricating oil distribution and the increase of the velocity of oil/gas two-phase flow, but it can improve the uniformity of heat transfer. In the vented bearing chamber, the thickness of oil film decreases with increase of rotor speed, and the speed of oil/gas two-phase flow increases greatly, and the entraining effect of air is obvious. The research results can provide a theoretical reference for improving the number and structure of bearing chambers to improve the performance of lubricating oil system.
Experimental study of the expansion of soil wetting fronts from a point source in heterogeneous sloping lands
Authors: Azizi Naser Ramzanian , Tashakori Askari , Maroufpoor Eisa , Emamgholizadeh Samad
Published: June 2023

This research intended to measure wetting front advancements (WFA) under various conditions using a physical model. Wetting front advance, along with its distribution in soils, is one of the important parameters in drip irrigation. It is influenced by many factors, including land slope, emitter discharge and soil texture. The effects of these factors were investigated by constructing a physical model with dimensions of 60 cm in width, 120 cm in height and 160 cm in length. The experiments were conducted using two heterogeneous soils, three land slopes (0, 10 and 20%), three emitter discharges (2, 4 and 8 L/h) and a constant volume of irrigation water (24 L). The results of the heterogeneous soils with three horizontal layers were also compared with those of three homogeneous soils (heavy, medium, and light textures). The results indicate that on sloping lands, the wetted area of the WFA downstream from the emitter was, on average, 20–62% greater than upstream from the emitter. With increases in land slope, the wetted depth under the emitter decreased by 3–18%. Also, when land slope changed from 0–10% and then again from 10–20%, the maximum radius of the wetting front increased, on average, by 32%, 44.8% and 77.5% for discharges of 2, 4 and 8 L/h, respectively.
དུས་གསུ་སྤྱི་བྱེ་བྲག་གི་རྣམ་གཞག་ལུ་དཔྱད་པ་ཐོས་ཆུང་བུའི་གཏམ།
Authors: རྡོ་རྗེ། ཕུར་བ་
Published: June 2023

དུས་གསུམ་སྤྱི་བྱེ་བྲག་གི་རྣམ་གཞག་བཤད་ཐངས་མ་འདྲཝ་ཡོདཔ་སྦེ་མཐོང་མི་ཚུ་ མཁས་པ་ག་གི་འདོད་པ་ར་ ཨིན་ན་དང་ འདོད་པ་མ་འདྲཝ་ག་དེམ་ཅིག་ཡོད་ག་ དེ་ཚུ་གི་བར་ན་ཁྱད་པར་ག་དེ་སྦེ་ར་ཡོདཔ་ཨིན་ན་ཚུ་ཁ་གསལ་ སྦེ་ཤེས་ཚུགས་ནི་གི་དོན་ལུ་དམིགས་གཏད་བསྐྱེད་དེ་ ཞིབ་འཚོལ་འདི་འབད་ཡོདཔ་ཨིན། ཞིབ་འཚོལ་ཐབས་ལམ་ དེ་ཡང་ ཁུངས་བཙན་ཐབས་ཤེས་ཀྱི་མཁོ་ཆས་ ཡིག་ཆ་དབྱེ་དཔྱད་དང་ དྲི་བ་དྲིས་ལན་གཉིས་ལག་ལེན་འཐབ་ཡོད། ཡིག་ཆ་འདི་ཡང་ བྱ་ཚིག་དུས་གསུམ་གྱིི་སྐོར་ལས་བཤད་པའི་དཔེ་དེབ་གསརཔ་དང་རྙིངམ་ཚུ་བསྡུ་སྒྲིག་ འབད་ཞིནམ་ལས་དབྱེ་དཔྱད་འབད་ཡོདཔ་དང་། དྲི་བ་དྲིས་ལན་འདི་ཡང་འབྲེལ་ཡོད་སློབ་གྲྭ་དང་བཤད་གྲྭའི་སློབ་ ཕྲུག་ ཟུར་གནས་མཁས་དབང་ཚུ་དང་འབྲེལ་བ་འཐབ་སྟེ་འབད་ཡོད། དེའི་སྐབས་སུ་ དུས་གསུམ་སྤྱི་བྱེ་བྲག་གི་ རྣམ་གཞག་འདོད་མི་མ་འདྲཝ་ཡོད་མི་ཚུ་ ཚུར་བསྡུ་བ་ཅིན་ འདོད་པ་མ་འདྲཝ་ཁག་བཞི་འདུག། དེ་ཡང་དུས་གསུམ་ རབ་གསལ་འོད་སྣང་ལྟར་དུ་གཅིག་དང་། ཉན་ཤུལ་མཁྱེན་རབ་ཀྱི་འདོད་པ་ལྟར་དུ་གཅིག། དེ་ལས་ནོར་བུ་བསྟན་ འཛིན་གྱིི་འདོད་པ་ལྟར་དུ་གཅིག་དང་ སྨོན་ལམ་ཚིག་མཛོད་ཀྱི་འདོད་ཚུལ་གཅིག་སྦེ་ཨིན་པས། དེ་ཡང་ དུས་ གསུམ་སྤྱི་བྱེ་བྲག་གི་འདོད་ཚུལ་མ་འདྲཝ་ཡོད་མི་ཚུ་ ཧིང་སང་ས་སྦེ་བཀོདཔ་ལས་ མ་འོངས་པ་ན་དེའི་སྐོར་ལས་ ལྷབ་སྦྱང་འབད་མི་ཚུ་མགུ་མ་འཐོམས་པར་ལྷབ་སྦྱངས་འབད་ནི་ལུ་སྟབས་བདེ་ནི་དང་། དེ་ལུ་བརྟེན་ཏེ་བྱ་ཚིག་གི་རྣམ་ གཞག་ལུ་མཁས་པ་འགྱུར་ནི་གི་འོས་འབབ་ཡོདཔ་ཨིན།
Increased Production and Flavonoids of Two Celery Highland Varieties (Apium graveolens L.) by Endophytic Bacteria in Lowland
Authors: Iman Andika Akbarul , Aziz Sandra Arifin , Munif Abdul
Published: June 2023

Celery (Apium graveolens L.) is an annual plant that grows optimally in Indonesia’s highlands. The provision of endophytic bacteria has the role of plant protection, increasing agronomic growth and plant physiology, and overcoming environmental stresses. This study aims to assist the adaptation of highland celery varieties in the lowlands due to the decreasing agricultural highland and increase their production with endophytic bacteria applications. The experiment was executed in Bogor, Indonesia using a randomized block design with two factors. The celery variety served as the first factor [Amigo and Aroma (highland variety), and Summer Green (lowland variety)], and endophytic bacteria was the second factor (control, APE35, a combination of APE35+BAT, APE35+BAT+EQ26, and the PTM3 consortium). The result showed that endophytic bacteria helped the plants to adjust growth from highland to lowlands and increase yield. In lowland, Amigo showed shorter in height but it has a significantly higher stalk number, carotenoids, and flavonoids than Summer Green (lowland control). Endophytic bacteria increased plant growth, N, P, pigments, and flavonoids. APE35 or PTM3 with Amigo produced significantly higher leaf area, number of tillers, number of leaflets, leaf area, head fresh weight (78.529 and 75.054 g/plant), total fresh weight (81.67 and 85.395 g/plant), and total flavonoids (1484.818 and 1502.459 mg QE/100 g) than Summer Green without endophytic bacteria.
Effect of Gamma Rays Irradiation to Cipedak Avocado Genetic Diversity
Authors: Ihsan Farihul , Ashari Sumeru , Soegianto Andy , Sukartini  , Affandi
Published: June 2023

Avocado cv. Cipedak is a superior cultivar that match the preferences of Indonesian consumers. However, it has several weaknesses, such as low edible portion, thin skin, and short fruit shelf life. One effort to improve the character weakness is by mutation breeding using gamma ray irradiation. Induction of mutation by gamma ray irradiation at a dose of 10 Gy was conducted on Cipedak avocado shoots in October 2021 which produced 13 first generation avocado mutants (M1). Observation on leaves morphological characters and DNA analysis were needed to find out the changes level of genetical diversity. The research was conducted in Cukurgondang Research Station and Molecular Laboratory of Indonesian Tropical Fruits Research Institute from January to March 2022. The plant material used involved 13 M1 progenies and cv. Cipedak. The material used were 20 weeks old grafted plants. The observed variables were macro and micromorphological characters as well as DNA-RAPD analysis. The results showed that there were phenotypic and genotypic changes in 13 M1 avocados seedlings. The coefficients of differences on macro and micromorphological characters were up to 33% and 28%, respectively and the coefficient of difference for DNA-RAPD analysis was 74%.
Instructions for CytoFLEX “Ready to Use” (RTU) QC Calibration Fluorospheres v1
Authors: C Tijerina Jamie
Published: June 2023

The CytoFLEX "Ready To Use" (RTU) QC Calibration Fluorospheres do not need to be diluted which reduces QC failures that are related to bead preparation errors. This protocol has been tested in the Flow Cytometry and Cell Sorting Facility at Caltech to ensure optimal results during QC in both plate and tube mode, while minimizing waste through reduction of the amount of beads that are discarded following QC, and minimizing plastic waste. Testing determined that one drop of beads deposited from the bottle is sufficient to run one test, however there must be enough placed in the tube or well to run a QC. In plate mode, the suggested minimum sample volume per well is 45ul. In tube mode, the suggested minimum sample volume is 100ul. These beads should be stored in a standard refrigerator close to the instrument. This protocol and workflow can be used in a core facility, or by an individual lab that has its own CytoFLEX instrument.
Correlation of serum interleukin-6 levels and neutrophil-lympocyte ratio in the severity of COVID-19 v1
Authors: Esa Budu Tenri
Published: June 2023

Background:Interleukin-6 (IL-6) is a pro-inflammatory cytokine that is produced at varying levels in patients with coronavirus disease 2019 (COVID-19). The neutrophil–lymphocyte ratio (NLR) is one of the new inflammatory markers of COVID-19. This study aimed to evaluate the differences in IL-6 level and the NLR in mild and severe COVID-19 and assess their correlation with COVID-19 severity and the correlation of IL-6 and NLR in COVID-19. Methods:A total of 91 patients with COVID-19 were divided into mild (n = 57) and severe (n = 34) COVID-19 groups. IL-6 levels were measured using the electrochemiluminescence immunoassay method on Roche Cobas e411. The NLR was the ratio of the total neutrophil and lymphocyte counts from complete haematology on the Sysmex XS-800i. Data were analysed using the Kolmogorov–Smirnov, Mann–Whitney, receiver operating characteristic curve, chi-square and Spearman correlation tests. The statistical test was significant at p <0.05. Results: Serum IL-6 levels and NLR significantly differed in mild and severe COVID-19. The median (min–max) IL-6 levels for mild and severe COVID-19 were 3.59 (1.50–638.30) pg/mL and 28.82 (5.52–926.30) pg/mL, respectively (p <0.001). The median (min–max) NLR in mild and moderate COVID-19 was 2.18 (0.69–15.58) and 8.13 (2.24–30.90), respectively (p <0.001). The obtained cut-off values for IL-6 and NLR were >6.99 pg/mL and >4.18, with odds ratios of 29.29 and 26.19, respectively. A positive correlation was found between IL-6 and NLR and COVID-19 severity (r = 0.612; p <0.001). Conclusions:The results indicated that serum IL-6 levels and NLR are higher in severe COVID-19 than in mild COVID-19. Patients with IL-6 levels >6.99 pg/mL and NLR >4.18 are 29 and 26 times more likely to suffer from severe COVID-19, respectively. Serum IL-6 levels and NLR are strongly correlated with COVID-19 severity. Serum IL-6 levels correlate with NLR in COVID-19.
Mitochondrial Antigen Presentation in RAW macrophages v2
Authors: Fahmy Ahmed
Published: June 2023

This protocol details methods for 3-day Mitochondrial Antigen Presentation Assay in a murine macrophage cell line (RAW) that expresses a glycoprotein B (gB) of herpes simplex virus 1 (HSV1) targeted to the mitochondrial matrix (mito-gB). A gB-specific CD8+ Tcell hybridoma recognizing the gB498–505peptide loaded on MHC class I molecules is also used to monitor antigen presentation through a beta-galactosidase assay kit.