Before revision:  Specifically, in pre-training phase , we first use a ranking-based BFS strategy to form the input node sequence. Then inspired by BERT, we adopt deep bi-directional transformer encoders to train the model, which is a variant of GNN aggregator that is more powerful than traditional deep neural networks like CNN and LSTM. The model is pre-trained based on two tasks, i.e., masked node modeling (MNM) and adjacent node prediction (ANP).
After revision:  Specifically, in pre-training phase , we first transform the neighborhood of a given node into a sequence. PF-HIN is pre-trained based on two tasks, i.e., masked node modeling (MNM) and adjacent node prediction (ANP).