Paper:
A Synaptic Neural Network (SynaNN) consists of synapses and neurons. Inspired by the synapse research of neuroscience, we built a synapse model with a nonlinear synapse function of excitatory and inhibitory channel probabilities. Introduced the concept of surprisal space and constructed a commutative diagram, we proved that the inhibitory probability function -log(1-exp(-x)) in surprisal space is the topologically conjugate function of the inhibitory complementary probability 1-x in probability space. Furthermore, we found that the derivative of the synapse over the parameter in the surprisal space is equal to the negative Bose-Einstein distribution. In addition, we constructed a fully connected synapse graph (tensor) as a synapse block of a synaptic neural network. Moreover, we proved the gradient formula of a cross-entropy loss function over parameters, so synapse learning can work with the gradient descent and backpropagation algorithms. In the proof-of-concept experiment, we performed an MNIST training and testing on the MLP model with synapse network as hidden layers.

Review:
Thanks for submitting your paper. It takes a lot of effort and courage to put your ideas out into the world. Sometimes the hardest work for researchers is conveying their thoughts to others in a manner in which those ideas can be understood.
With that in mind, I had an extremely difficult time following your arguments. 
I noticed several things:
 - There are numerous places in the text that lack proper citation, or are cited improperly.
 - Why was there not a related methods section? I find it hard to believe that all of your ideas have no precursor.
 - When there are citations, there is usually only one text and it is quite old. For example, all of your neuroscience citations reference a work that is almost 40 years old. There have been quite a few improvements in our biological understanding as well as theoretical understanding since then. ( I make this point as a common justification used in the manuscript is that the method describes how synapses function in biology. )
- There is a claim regarding how this can be used in fintech. This statement doesn't belong in this work.
 - There are many different equations given throughout the text. Some of these equations come from areas like physics or information theory, and others seem to be of your own design. Regarding the latter, there is no justification or explanation for the origin of the equations. Regarding the former, if you are using equations from lots of different fields, or even field you think part of your audience might not be familiar with, you should, at the very least, include a some description of the algorithm or intuition as to why it is being leveraged.
 - It wasn't clear from your diagrams or your descriptions what the difference between a synapse and a neuron was in your architecture. It seemed like the name was used interchangeably in some areas, but then had a strict definition in others.
 - I was also not able to understand how the excitatory and the inhibitory connections that were to enter each neuron were connected to the previous layer of the network. Is a link between neurons in Figure 2 actually two links? If this is the case, then it is a direct violation of Dale's law. Again, I only mention this because most arguments seem to be of the form "this is correct because it is how it is done biologically".
 - There were a few claims made in the paper that were completely unsubstantiated. A good example of this was in the conclusion section part ii) where it was stated that "using a large number of synapses and neurons SynaNN can solve the complex problems in the real world."
- Also, the last sentence of the conclusion was not discussed anywhere in the rest of the paper. Nor was the statement itself supported except with a single citation and no description.
Regarding the empirical testing of your algorithm, I was very dissapointed to see that the only dataset it was tested against was MNIST. Furthermore there was absolutely no benchmarking against other comparative algorithms. At the very least I would have expected a comparison to the perceptron algorithm that you use as inspiration, but that would also still not have been enough.
This paper needs heavy amounts of work to make it understandable. Once it is understandable an attempt to evaluate the merit of the scientific contribution would then be possible.