## 4 Solution for the Linearly Inseparable XOR Problem  
A neural network can be described as a programmatic system which allows making decisions using the evolution of a complex non-linear system. Such decisions are based on recognized underlying relationships in datasets. Neural networks take in a vector encoding the input object; the output signal of a neural network encodes the decision made by the system [38]. A perceptron is a mathematical model, proposed by [PERSON] in 1957; it can be treated as a simple neural network used to classify the data into two classes. Neural networks are often trained using back propagation, however, evolutionary algorithms show promising results as well [39; 40; 41]. As shown in [26], a hybrid algorithm which incorporates evolutionary-inspired and classical optimization techniques can achieve faster convergence speed and better accuracy. The FSSGD, as the most effective hybrid algorithm proposed in this paper, can be used to optimize loss functions of neural networks, in cases, when the loss function of a perceptron is differentiable inside the study region. As a proof-of-concept, we used FSSGD as the loss function optimization algorithm to train a multi-layer perceptron. We were solving the linearly inseparable exclusive disjunction (XOR) problem [42], commonly used to benchmark neural network optimization algorithms [43]. In the problem we consider, we have the \(\mathbb{X}\) set of input objects, and the \(\mathbb{Y}\) set of answers. Both the objects and the answers are Boolean values, belonging to the {0, 1} set. The goal is to restore the function \(f^{*}:\mathbb{X}\rightarrow\mathbb{Y}\) using a multi-layer perceptron. The binary XOR function can be described by Table 7.  
To solve the formulated problem, we built a perceptron with one hidden layer consisting of eight neurons. Other neural network architectures exist, which can solve the XOR problem [43; 44], but the goal of the experiment was to see how the FSSGD algorithm would deal with a plenty of conflicting solutions. Perceptron architecture is shown in Figure 3.  
On the hidden layer, we used the hyperbolic tangent activation function, given by:  
\[th(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}. \tag{10}\]  
where, \(x\) is the input signal multiplied by the weight value of a neuron. While the \(\mathbb{Y}\) set of decisions of our neural network should be represented as the Boolean values, we used the sigmoid activation function on the output layer. The sigmoid activation function is given by:  
\[\sigma(x)=\frac{1}{1+e^{x}}. \tag{11}\]  
We used the binary cross-entropy as the loss function for the multi-layer perceptron:  
\[L\Big{(}\mathbb{X},\mathbb{Y},\overset{\rightarrow}{w}\Big{)}=-\frac{1}{N} \sum\
olimits_{i=0}^{N}\Big{(}y_{i}\log f\Big{(}\overset{\rightarrow}{x}_{i},\overset{\rightarrow}{w}\Big{)}+(1-y_{i})\log\Big{(}1-f\Big{(}\overset{ \rightarrow}{x}_{i},\overset{\rightarrow}{w}\Big{)}\Big{)}\Big{)}. \tag{12}\]  
where, \(\mathbb{X}\) and \(\mathbb{Y}\) are the set of objects and the set of answers, respectively; \(\overset{\rightarrow}{w}\) denotes the weights; \(y_{i}\) is the \(i\)-th answer, \(y_{i}\in\mathbb{Y}\); \(\overset{\rightarrow}{x}_{i}\) is the \(i\)-th object, \(\overset{\rightarrow}{x}_{i}\in\mathbb{X}\); \(N=[\mathbb{X}]=[\mathbb{Y}]\). \begin{table}
\begin{tabular}{c c c} \hline \hline \(\mathbf{X}_{1}\) & \(\mathbf{X}_{2}\) & \(\mathbf{Y}\) \\ \hline
1 & 1 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Exclusive disjunction (XOR) function truth table. The training process of a neural network leads to the optimization of its loss function. In our case, we optimized the Equation (12) loss function, using the hybrid FSSGD algorithm, in respect to the weights vector \(\overrightarrow{w}\). After the neural network loss function was optimized, we plotted the surface of the loss function using the method [45], which allows visualizing functions that live in very high-dimensional spaces.