# Optimization Techniques for Design Problems in Selected Areas in WSNs: A Tutorial
## 5 Fundamentals of Nonlinear Optimization  
This section provides the theoretical discussion on nonlinear programs and their optimality conditions. It forms the theoretical basis of the development of several algorithms for nonlinear optimization. These techniques are illustrated Figure 9, which is a diagram for the taxonomy of those techniques. A nonlinear programming (NLP) problem deals with the search of a minimum of a function \(f\left(\mathbf{x}\right)\) of \(n\) real variables \(\mathbf{x}=\left(x_{1},x_{2},...,x_{n}\right)\in\mathbb{X}\subseteq\mathbb{ R}^{n}\) subject to the set of equality constraints \(\mathbf{h}\left(\mathbf{x}\right)=\mathbf{0}\) (\(h_{i}\left(\mathbf{x}\right)=0\), \(i=1,2,...,m\)), and a set of inequality constraints \(\mathbf{g}\left(\mathbf{x}\right)\leq\mathbf{0}\) (\(g_{i}\left(\mathbf{x}\right)\leq 0,\ j=1,2,...,p\)) and is denoted as:  
\[min\ f\left(\mathbf{x}\right)\] (6a) s.t. \[\mathbf{h}\left(\mathbf{x}\right) =\mathbf{0} \tag{6b}\] \[\mathbf{g}\left(\mathbf{x}\right) \leq\mathbf{0}\] (6c) \[\mathbf{x} \in\mathbb{X}\subseteq\mathbb{R}^{n} \tag{6d}\]  
If any of the functions in Equation (6) is nonlinear, then Equation (6) is an NLP. For the sake of our discussion, we assume that those functions are continuous and twice differentiable. We list the following important definitions that are commonly referred to in any discussion on NLPs:  
1. **Feasible points**: a point \(\mathbf{x}\in\mathbb{X}\) satisfying all the constraints in Equation (6) is a feasible point and is defined as \[F=\left\{\mathbf{x}\in\mathbb{X}\subseteq\mathbb{R}^{n}:\mathbf{h}\left( \mathbf{x}\right)=\mathbf{0},\mathbf{g}\left(\mathbf{x}\right)\leq\mathbf{0}\right\}\]
2. **Active, inactive constraints**: An inequality constraint \(g_{j}\left(\mathbf{x}\right)\) is called active at a feasible point \(\bar{\mathbf{x}}\in\mathbb{X}\) if \(g_{j}\left(\mathbf{x}\right)=0\), and is called inactive at a feasible point \(\bar{\mathbf{x}}\in\mathbb{X}\) if \(g_{j}\left(\mathbf{x}\right)<0\). The constraints that are active at a feasible point \(\bar{\mathbf{x}}\) restrict the feasibility domain while the inactive constraints do not impose any restrictions on the feasibility in the neighborhood of \(\bar{\mathbf{x}}\), defined as a ball of radius \(\epsilon\) around \(\bar{\mathbf{x}}\), \(B_{\epsilon}\left(\bar{\mathbf{x}}\right)\). 3. **Local minimum:**\(\mathbf{x}^{*}\in F\) is a local minimum of Equation (6) if there exists a ball of radius \(\epsilon\) around \(\mathbf{x}^{*}\), \(B_{\epsilon}\left(\bar{\mathbf{x}}\right)\): \[f\left(\mathbf{x}^{*}\right)\leq f\left(\mathbf{x}\right)\quad\forall\ \ \mathbf{x}\in B_{\epsilon}\left(\mathbf{x}^{*}\right)\cap F\]
4. **Global minimum:**\(\mathbf{x}^{*}\in F\) is a global minimum of Equation (6) if \[f\left(\mathbf{x}^{*}\right)\leq f\left(\mathbf{x}\right)\quad\forall\ \ \mathbf{x}\in F\]5. **Feasible direction vector**: Let a feasible point \(\bar{\mathbf{x}}\in F\). Then any point \(\mathbf{x}\) in a ball of radius \(\epsilon\) around \(\bar{\mathbf{x}}\) which can be written as \(\bar{\mathbf{x}}+\mathbf{d}\) is a nonzero vector if and only if \(\mathbf{x}\
eq\bar{\mathbf{x}}\). A vector \(\mathbf{d}\
eq\mathbf{0}\) is called a feasible direction vector from \(\bar{\mathbf{x}}\), if there exists a ball of radius \(\epsilon\): \[\left(\bar{\mathbf{x}}+\lambda\mathbf{d}\right)\in B_{\epsilon}\left(\bar{ \mathbf{x}}\right)\cap F\quad\forall\quad 0\leq\lambda\leq\frac{\epsilon}{\| \mathbf{d}\|}\] The set of feasible directions vectors \(\mathbf{d}\
eq\mathbf{0}\) from \(\bar{\mathbf{x}}\) is called the _cone_ of feasible directions of \(F\) at \(\bar{\mathbf{x}}\). An important remark is that if \(\bar{\mathbf{x}}\) is a local minimum of Equation (6) and \(\mathbf{d}\) is a feasible direction vector from \(\bar{\mathbf{x}}\), then for sufficiently small \(\lambda\) we must have \[f\left(\bar{\mathbf{x}}\right)\leq f\left(\bar{\mathbf{x}}+\lambda\mathbf{d} \right).\]
6.