Package: sinabs-exodus
sinabs-exodusSinabs-exodus is a plugin to thesinabsspiking neural network library. It can provide massive speedups in training and inference on GPU.The tool is based onEXODUS, a formulation of backpropagation-through-time with surrogate gradients, that allows for efficient parallelization. EXODUS stands forEXact calculationOfDerivatives asUpdate toSLAYER. It builds upon the SLAYER[^1] algorithm, but uses mathematically accurate gradients and tends to be more robust to surrogate gradient scaling, making training less prone to suffer from exploding or vanishin gradients.Some of the code in this library is loosely based uponslayerPytorch, the python implementation of SLAYER.If you use any of this code please cite the following publication:@article{bauer2022exodus,
  title={EXODUS: Stable and Efficient Training of Spiking Neural Networks},
  author={Bauer, Felix Christian and Lenz, Gregor and Haghighatshoar, Saeid and Sheik, Sadique},
  journal={arXiv preprint arXiv:2205.10242},
  year={2022}
}Getting startedPrerequisitesEXODUS uses CUDA for efficient computation, so you will need a CUDA-capable GPU, and a working installation ofCUDA.If you have CUDA installed, you can use the command$ nvcc -Vto see the installed version. The last line should say something likeBuild cuda_xx.x....., where x.xx is the version.
Note that$ nvidia-smidoesnotshow you the installed CUDA version, but only the newest version your Nvidia driver is compatible with.You should also make sure that you have aPyTorchinstallation that is compatible with your CUDA version.
To verify this, open a python console and runimport torch
print(torch.__version__)The part after the+in the output is the CUDA version that PyTorch has been installed for and should match that of your system.InstallationAfter cloning this repository, the package can simply be installed via pip.
This is anamespace packagemeaning that once installed this will be sharing its namespace withsinabspackage.$ pip install .Do not install in editable (-e) mode.UsageIf you have used sinabs before, using EXODUS is straightforward, as the APIs are the same.
You just need to import the spiking or leaky layer classes that you want to speed up fromsinabs.exodus.layersinstead ofsinabs.layers.Supported classes are:IAFLIFExpLeakFor example, instead offrom sinabs.layers import IAF

iaf = IAF()dofrom sinabs.exodus.layers import IAF

iaf = IAF()Conversion to and from Sinabs classesEXODUS provides convenience functions for converting EXODUS objects to their counterparts in Sinabs and vice versa in thesinabs.exodus.conversionmodule. In the following example, a new objectexodus_modelis created that is the same assinabs_model, but with all sinabs-based layers being replaced with EXODUS equivalents, where possible. The originalsinabs_modelcan be anytorch.nn.Moduleobject. Currently, classes that can be converted to and from EXODUS are:IAF,IAFSqueeze,LIF,LIFSqueeze,ExpLeak, andExpLeakSqueeze.from torch.nn import Sequential, Conv2d, AvgPool2d
from sinabs.layers import IAF
from sinabs.exodus import conversion

# This could be any torch module
sinabs_model = Sequential(Conv2d(3, 4, 1), AvgPool2d(2), IAF())

# Convert sinabs layers to exodus layers
exodus_model = conversion.sinabs_to_exodus(sinabs_model)Converting from EXODUS to Sinabs:new_sinabs_model = conversion.exodus_to_sinabs(exodus_model)Frequent IssuesCUDA is not installed or version does not match that of torchIf during installation you get an error, such asRuntimeError:
The detected CUDA version (...) mismatches the version that was used to compile
PyTorch (...). Please make sure to use the same CUDA versions.orOSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.CUDA is either not installed properly on your system or the version does not match that of torch (seeabove).
If you do have the correct version installed and the error still comes up, try to make sure that the environment variables such asPATHandLD_LIBRARY_PATHcontain references to the correct directories. Please refer to NVIDIA's installation instructions for more details on how to do this for your system.The same holds if, while using EXODUS, you get an error like:undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefIlEENS2or similar.LicenseSinabs-exodus is published under AGPL v3.0. See the LICENSE file for details.Footnotes[^1]: Sumit Bam Shrestha and Garrick Orchard. "SLAYER: Spike Layer Error Reassignment in Time."
InAdvances in Neural Information Processing Systems, pp. 1417-1426. 2018.