Embedding of dynamical networks
Creators: name: Radmanesh, Mohammadreza
Publisher: name: RMIT University
Year: 2024
Subjects: classificationCode: 490105
subject: Dynamical systems in applications

description: Many real-world systems can be represented as networks, such as citation networks, ‎social networks, ‎biological networks, transportation networks, and power grids. Many of ‎these networks often experience exponential growth in their complexity. For example, in the ‎World Wide Web network, online social networks have witnessed tremendous growth, many ‎‎of which currently having billions of nodes and connections. Network calculations could ‎become a difficult task ‎on large-scale networks, calling for effective strategies to reduce ‎dimensionality of network data.‎ Network embedding aims to transform a network into a low-‎dimensional vector space while preserving the ‎structural characteristics of the network and ‎maintaining key features to reconstruct the network from embedded ‎vectors. This allows for ‎various insights and analytics applications to improve network analysis performance or ‎end ‎user experience. As such, network embedding techniques can be applied on a range of ‎downstream ‎tasks, such as link prediction, node classification and community detection. The ‎three common network embedding models include i) matrix factorization methods,‎ ii) ‎random walk approaches‎, and iii) neural network methods. In this thesis, due to many ‎advantages of neural network-based methods such as scalability and generalizability, all the ‎proposed methods are based on neural networks. Based on a network type and ‎characteristics, there are different challenges in network embedding models. Some of them ‎include dimensionality of the embedding, model scalability, model generalizability and high ‎computational complexity. As an instance, in directed networks, along with the stated ‎challenges, the embedding methods need to deal with asymmetric nature of the network or ‎in a biological network in the presence of a background noise. ‎

In this thesis, I address these challenges and some others related to the target network. ‎To show the effectiveness of our approaches, I conducted experiments on a wide range of ‎networks from social networks to citation networks to biological networks with different ‎characteristics including weighted or unweighted, directed or undirected, and small or large ‎scale. The experimental results show the superiority of proposed methods against state-of-‎the-art algorithms. First, I show how involving higher-order proximities can deal with ‎complex nature of the real-world networks by maximally ‎considering the topological ‎information in the optimisation process. Our proposed methods preserve the network’s ‎structural properties ‎by optimising higher-order proximity measures. The experimental ‎results show superior performance compared to baselines in terms of the network ‎reconstruction task. Furthermore, in terms of network calculation tasks, I compare our ‎method with other embedding algorithms on link prediction and multi-label classification.‎ ‎Our experimental results show that the optimal proximity order highly depends on the target ‎network and intended downstream task. ‎

Then, I study directed attributed networks by proposing a novel deep asymmetric ‎attributed network embedding ‎model based on the convolutional graph neural network, ‎called Asymmetric Attributed Graph Convolutional Network (AAGCN). The main idea is to ‎maximally ‎preserve the asymmetric proximity and asymmetric similarity of directed ‎attributed networks. AAGCN ‎introduces two neighbourhood feature aggregation schemes to ‎separately aggregate the features of a node ‎with the features of its in- and out- neighbours. ‎Then, it learns two embedding vectors for each node, one ‎source embedding vector and one ‎target embedding vector. The final representations are the results of ‎concatenating source ‎and target embedding vectors. I test the performance of AAGCN on three real-world ‎‎networks for network reconstruction, link prediction, node classification and visualization ‎tasks. The ‎experimental results show the superiority of AAGCN against state-of-the-art ‎embedding methods.‎

Finally, as an application to network embedding, I consider online spike sorting ‎problem. Spike sorting – the process of separating spikes from different neurons – is often ‎the first and most critical step in the neural data analysis pipeline. Despite several ‎advancements in this area, an important remaining challenge in neuroscience is online spike ‎sorting. Current approaches to online spike sorting are not fully automated, are ‎computationally expensive and are often outperformed by offline approaches. I present a ‎novel algorithm for fast and robust online classification of single neuron activity. This ‎algorithm is based on a deep contractive autoencoder (CAE) architecture. CAEs are neural ‎networks that can learn a latent state representation of their inputs. The main advantage of ‎CAE-based approaches is that they are less sensitive to noise (i.e., small perturbations in ‎their inputs). Overall, our deep CAE-based online spike sorting algorithm achieves over 90% ‎accuracy in sorting unseen spike waveforms, outperforming existing models and conserving ‎a performance close to the offline case. In the offline scenario, our method substantially ‎outperforms the existing models, providing an average improvement of 40% in accuracy ‎over different datasets. ‎