## 4 Mini-Batch Adaptive Search Algorithm  
Let us consider the optimization problem \(J_{d}[K_{d}]\rightarrow\underset{K_{d}}{\text{min}}\). Denote: \(J_{d}^{s}\) is the minimum value of the cost function after the \(s-\)th run; \(\hat{K}_{d}^{s}\) is a best parameter vector column after startup; \(d\) is the mini-batch size. Step 0. Set the initial mini-batch size: \(d=1\) (in general, one can start with any value of \(1\leq d\leq N\)); \(S_{\text{max}}\) is a maximum number of starts; \(B_{\text{max}}\) is a maximum number of passes; \(\alpha=1.618\) is an expansion coefficient; \(\beta=0.618\) is a compression coefficient; \(M\) is a maximum number of failed tests at the current iteration; \(t_{0}=1\) is an initial step size (one can use any value \(t_{0}>R\)), \(R\) is a minimum step size and \(L\) is a maximum number of iterations; and \(r\) is a number of initial trial solutions (\(1\leq r\leq 10\)). Step 1. Set the values:\(b=1\) (passes number counter) and \(P_{d}=0\)(initial value of the sum of the cost function average values). Step 2. Set the values: \(s=1\)(starts number counter) and \(J_{d}^{1}=10^{8}\div 10^{10}\); \(S_{d}=0\)(initial value of the sum of the objective function values). Step 3. Define the initial values of coefficients \(u_{t_{0},t_{1},\ldots,t_{n}}^{j}\), \(K_{t_{0},t_{1},\ldots,t_{n}}^{j}\). Generate \(r\) vectors \(K_{d}^{s}\) using a uniform distribution of its coordinates at some intervals. Calculate the value of the function \(J_{d}\) for each generated vector and order them according to the value of the objective function [17]. The vector with the smallest value of the objective function is denoted by column vector \(K_{d}^{s,0}\). Put \(l=0,j=1\). Step 4. Generate a random vector \(\xi_{d}^{j}=\left(\xi_{1}/\ldots,\xi_{q^{j}}\right)^{T}\), where \(\xi_{i}^{j}\) is a random variable uniformly distributed on the interval \([-1,1]\). Step 5. Calculate: \(y^{j}=K_{d}^{s,J}+t_{1}\frac{y^{j}}{\|\xi^{j}\|}\). Step 6. Generate the mini-batch size \(d\), i.e., generate \(d\) pairwise mismatched sets of \(q_{1}\in N_{1},\ldots,q_{n}\in N_{n}\) values defining the initial states \(X_{0}^{k}\in\Omega\) with numbers \(k=q_{1}\cdot\ldots\cdot q_{n}\in\{1,\ldots,N\}\) or corresponding to the tuple \(<q_{1},\ldots,q_{n}>\). Check the fulfillment of the conditions:  
1. If \(J_{d}(y^{j})<J_{d}(K_{d}^{s,J})\), the algorithm step is successful. Put \(z^{j}=K_{d}^{s,J}+\alpha(y^{j}-K_{d}^{s,J})\). Determine if current direction \(y^{j}-K_{d}^{s,J}\) is successful: if \(J_{d}(z^{j})<J_{d}(K_{d}^{s,J})\), the search direction is successful. Put \(K_{d}^{s,J+1}=z^{j}\), \(t_{1+1}=\alpha t_{l}\), \(l=l+1\) and check the termination condition. If \(l<L\), put \(j=1\) and go to step 4. If \(l=L\), the search process is over: \(\hat{K}_{d}^{s}=K_{d}^{s,J}\), go to step 8; if \(J_{d}(z^{j})\geq J_{d}(K_{d}^{s,J})\), the search direction is unsuccessful, go to step 7;
2. If \(J_{d}(y^{j})\geq J_{d}(K_{d}^{s,J})\), the unsuccessful step is made, go to step 7. Step 7. Calculate the number of unsuccessful steps from the current solution:  
1. If \(j<M\), put \(j=j+1\) and go to step 4;
2. If \(j=M\), check the termination condition: if \(t_{l}\leq R\), the process is over: \(\hat{K}_{d}^{s}=K_{d}^{s,J}\) and \(J_{d}^{s}=J_{d}(K_{d}^{s,J})\), go to step 8; if \(t_{l}>R\), put \(t_{l}=\beta\)\(t_{l}\), \(j=1\) and go to step 4. Step 8. Check the improvement of the cost function value as a result of the \(s-\)th run: if \(J_{d}(K_{d}^{s,J})<J_{d}^{s}\), put \(J_{d}^{s}=J_{d}(K_{d}^{s,J})\) and \(\hat{K}_{d}^{s}=K_{d}^{s,J}\) and go to step 9; if \(J_{d}(K_{d}^{s,J})\geq\hat{J}_{d}^{s}\), go to step 9. Step 9.