Package: tf-siren
Tensorflow Sinusodial Representation Networks (SIREN)Tensorflow 2.0 implementation of Sinusodial Representation networks (SIREN) from the paperImplicit Neural Representations with Periodic Activation Functions.InstallationPip install$pipinstall--upgradetf_sirenPip install (test support)$pipinstall--upgradetf_siren[tests]UsageFor general usage equivalent to the paper, import and use eitherSinusodialRepresentationDenseorSIRENModel.fromtf_sirenimportSinusodialRepresentationDensefromtf_sirenimportSIRENModel# You can use SinusodialRepresentationDense exactly like you ordinarily use Dense layers.ip=tf.keras.layers.Input(shape=[2])x=SinusodialRepresentationDense(32,activation='sine',# default activation functionw0=1.0)(ip)# w0 represents sin(w0 * x) where x is the input.model=tf.keras.Model(inputs=ip,outputs=x)# Or directly use the model class to build a multi layer SIRENmodel=SIRENModel(units=256,final_units=3,final_activation='sigmoid',num_layers=5,w0=1.0,w0_initial=30.0)For the(experimental)kernel scaled variants, import and use eitherScaledSinusodialRepresentationDenseorScaledSIRENModel.fromtf_sirenimportScaledSinusodialRepresentationDensefromtf_sirenimportScaledSIRENModel# You can use SinusodialRepresentationDense exactly like you ordinarily use Dense layers.ip=tf.keras.layers.Input(shape=[2])x=ScaledSinusodialRepresentationDense(32,scale=1.0# scale value should be carefully chosen in range [1, 2]activation='sine',# default activation functionw0=1.0)(ip)# w0 represents sin(w0 * x) where x is the input.model=tf.keras.Model(inputs=ip,outputs=x)# Or directly use the model class to build a multi layer Scaled SIRENmodel=ScaledSIRENModel(units=256,final_units=3,final_activation='sigmoid',scale=1.0,num_layers=5,w0=1.0,w0_initial=30.0)Results on Image Inpainting taskA partial implementation of the image inpainting task is available as thetrain_inpainting_siren.pyandeval_inpainting_siren.pyscripts inside thescriptsdirectory.Weight files are made available in the repository under theReleasetab of the project. Extract the weights and place thecheckpointsfolder at the scripts directoryThese weights generates the following output after 5000 epochs of training with batch size 8192 while using only 10% of the available pixels in the image during training phase.If we train for using only 20% of the available pixels in the image during training phase -If we train for using only 30% of the available pixels in the image during training phase -SIREN Hyper NetworkWe can use a Hyper Network in order to encode an entire dataset into the weights of a SIREN model. The weights for the SIREN model are generated by this hyper network, which computes these weights based on an encoded representation.Support for the Hyper Network is available by usingNeuralProcessHyperNet, which uses theSetEncoderfrom the paper as the encoder.Training on the CIFAR 10 dataset is available inside thescriptsdirectory -train_cifar_inpainting_siren.pyandeval_cifar_inpainting_siren.py.Pre-trained weights are available in theReleasetab underassets.On evaluating on the test set with 1000 context pixels, this model gets an average MSE of0.009. Using 100 context pixels, the MSE increases to0.019.The following image is using 1000 context pixels on the test set :(Experimental) Comparison of convergence between original and kernel scaled SIRENsThe kernel scaled variants of the model converge faster than the original SIREN under certain circumstances. All the models below are trained with Adam optimizer with constant learning rate of 5e-5 for 5000 epochs and batch size of 8192 on the same image pixels (10% of the celtic spiral image).The tensorboard logs can be found here -Scale = 1.0Scale = 1.2Scale = 1.5Scale = 1.8Scale = 2.0Citation@inproceedings{sitzmann2019siren,
    author = {Sitzmann, Vincent
              and Martel, Julien N.P.
              and Bergman, Alexander W.
              and Lindell, David B.
              and Wetzstein, Gordon},
    title = {Implicit Neural Representations
              with Periodic Activation Functions},
    booktitle = {arXiv},
    year={2020}
}RequirementsTensorflow 2.0+Matplotlib to visualize eval result