Single-Layer Learnable Activation for Implicit Neural Representation
  (SL^{2}A-INR)
Authors: name: Moein Heidari; name: Reza Rezaeian; name: Reza Azad; name: Dorit Merhof; name: Hamid Soltanian-Zadeh; name: Ilker Hacihaliloglu
Published: 2024-09-17 02:02:15

Abstract: Implicit Neural Representation (INR), leveraging a neural network to
transform coordinate input into corresponding attributes, has recently driven
significant advances in several vision-related domains. However, the
performance of INR is heavily influenced by the choice of the nonlinear
activation function used in its multilayer perceptron (MLP) architecture.
Multiple nonlinearities have been investigated; yet, current INRs face
limitations in capturing high-frequency components, diverse signal types, and
handling inverse problems. We have identified that these problems can be
greatly alleviated by introducing a paradigm shift in INRs. We find that an
architecture with learnable activations in initial layers can represent fine
details in the underlying signals. Specifically, we propose SL^{2}A-INR, a
hybrid network for INR with a single-layer learnable activation function,
prompting the effectiveness of traditional ReLU-based MLPs. Our method performs
superior across diverse tasks, including image representation, 3D shape
reconstructions, inpainting, single image super-resolution, CT reconstruction,
and novel view synthesis. Through comprehensive experiments, SL^{2}A-INR sets
new benchmarks in accuracy, quality, and convergence rates for INR.