A super tiny mnist model to demostrate the potential of the learnable activation - OptAEG-V1.
The model can reach 96.6% accuracy with only 1.3k parameters.
The OptAEG-V1 learnable activation is based on a theory of Arithmetic Expression Geometry which is still in developing. Please visit the draft papers on theory and neural networks for a reference