## 2 The Proposed Model and Strategies
### Integer-Order Neural Network Models in Product Form Design  
One objective of this study is the extension of some existing neural network models in product form design. As a bio-inspired approach, ANNs attempt to mimic some of the learning abilities of the human brain. They are composed of elements that process information depending on the received signals and their weights and are located in several layers. A typical neural network model consisting of three layers (input, hidden and output) is demonstrated in Figure 1[3]. Starting from the input layer, the network elements propagate the signals from layer to layer until the outputs are generated by the output layer. Various activation functions, including linear, sigmoid and tangent (Figure 2), are applied to generate the outputs of the neurons in the hidden layer and the outputs of the neurons in the output layer. In reviewing the available results related to the use of neural networks in industrial form design, in most of the constructed neural network models, the input variables are the initial design concepts, while the output variables represent the optimal design solution. For example, the ANN used in [7] for product design with respect to the product form and product image can be represented by the equation  
\[y_{k}=f\left(\sum_{j=1}^{m}w_{jk}x_{j}-\theta_{k}\right),\;1\leq k\leq p,\]  
where \(y_{k}\) are the outputs, \(x_{j}\) are the input variables, \(f(\cdot)\) is the activation function, \(w_{jk}\), \(j=1,2,\ldots,m\), \(k=1,2,\ldots,p\) are the connection weights, and \(\theta_{k}\) are threshold values. The neural network model used to predict personal design preferences in form finding in [11] can be represented by the equation  
\[\hat{y}=Sigmoid(w\ast x+b),\]  
where \(\hat{y}\) is the value in the current neuron, \(x\) is the value in the previous neuron(s), _Sigmoid_ is the sigmoid activation function, and \(w\) and \(b\) are the parameters, which represent the learning activity of the network. Figure 1: A three layer neural network model [3]. Figure 2: Commonly used activation functions and their graphs [3]. The generalization of some discrete ANN models used in engineering design has been proposed in [51] by the discrete-time neural network  
\[z_{h}(t+1)=-a_{h}z_{h}(t)+\sum_{j=1}^{n}w_{hj}f_{j}(z_{j}(t))+\Theta_{h},\;1\leq h \leq n, \tag{1}\]  
where \(n\) is the number of elements in the neural network model (1), \(z_{j}(k)\) is the state of the input \(j\) at the discrete time \(t\), \(t=1,2,\ldots\), \((z_{1}(t),z_{2}(t),\ldots,z_{n}(t))^{T}\)\(w_{hj}\) are the connection weights, \(f_{j}(z_{j}(t))\) are the activation functions corresponding to different neurons \(j\) (\(j=1,2,\ldots,n\)), and \(\Theta_{h}\) is the external bias of the neuron \(h\). The generalized model (1) takes into account the ability to consider different activation functions for different neurons. Moreover, constants \(a_{h}>0\) are added to consider the possibility of the neuron \(h\) (\(h=1,2,\ldots,n\)) resetting its potential to the resting state when isolated from other nodes and inputs. The initial conditions related to model (1) are in the form  
\[z_{h}(0)=z_{h}^{0},\;z_{h}^{0}\in\mathbb{R},\;h=1,2,\ldots,n. \tag{2}\]  
The existing literature supports the fact that ANN models of type (1) are extremely important in product form design analysis to explore the relationships between some initial design variables (inputs) \(z_{h}^{0}\) and output variables (outputs) \(z_{h}(t)\) at any stage \(t\) of the design process, and, finally, to predict the optimal design solution. The visualization presented in Figure 3 demonstrates the role of the activation function, connection weights and external biases at discrete time \(t\). Each design variable \(z_{h}\) at \(t+1\) receives signals from all previous variables \(z_{h}(t)\), which represent the design parameters (concepts) at \(t\). The input variables \(z_{h}(t)\) are activated by means of different (in general) activation functions. Each connection between the activated cells has a different weight.