Intelligent Matrix Exponentiation
Authors: Thomas Fischbacher, Iulia Maria Comsa, Krzysztof Potempa, Moritz Firsching, Luca Versari, Jyrki Alakuijala
Keywords: matrix exponential, tensor methods, supervised learning, domain extrapolation, certified robustness
Abstract: We present a novel machine learning architecture that uses a single high-dimensional nonlinearity consisting of the exponential of a single input-dependent matrix. The mathematical simplicity of this architecture allows a detailed analysis of its behaviour, providing robustness guarantees via Lipschitz bounds. Despite its simplicity, a single matrix exponential layer already provides universal approximation properties and can learn and extrapolate fundamental functions of the input, such as periodic structure or geometric invariants. This architecture outperforms other general-purpose architectures on benchmark problems, including CIFAR-10, using fewer parameters.