  Representation learning (RL) methods learn objects' latent embeddings where
information is preserved by distances. Since distances are invariant to certain
linear transformations, one may obtain different embeddings while preserving
the same information. In dynamic systems, a temporal difference in embeddings
may be explained by the stability of the system or by the misalignment of
embeddings due to arbitrary transformations. In the literature, embedding
alignment has not been defined formally, explored theoretically, or analyzed
empirically. Here, we explore the embedding alignment and its parts, provide
the first formal definitions, propose novel metrics to measure alignment and
stability, and show their suitability through synthetic experiments. Real-world
experiments show that both static and dynamic RL methods are prone to produce
misaligned embeddings and such misalignment worsens the performance of dynamic
network inference tasks. By ensuring alignment, the prediction accuracy raises
by up to 90% in static and by 40% in dynamic RL methods.
