## 1 Introduction
### Derivative-Based Methods  
Derivative-based methods, also known as gradient-based methods, utilize information about the derivative of the objective function to guide the search for the optimum. These methods are effective when the objective function is differentiable. In other words, derivative-based methods are particularly useful in continuous optimization problems where the objective function is smooth and the derivatives can be efficiently computed. Some derivative-based optimization algorithms include:  
1. **Gradient descent:** Iteratively updates the solution in the direction of the steepest descent of the objective function [17]. 2. **[PERSON]'s method:** Utilizes both the first and second derivatives of the objective function to approximate the optimum more efficiently [18]. 3. **Quasi-Newton methods:** Approximate the Hessian matrix (second derivatives) using a limited number of function and gradient evaluations to improve convergence speed [19]. It is important to note that this classification is not exhaustive, and there are other specialized optimization algorithms and techniques available for different types of problems. The choice of optimization algorithm depends on the problem characteristics, computational resources, and desired trade-offs between solution quality and computational efficiency [20]. In addition, optimization has diverse applications across various domains, such as engineering design, operations management, financial planning, scheduling, machine learning, and data analysis. It plays a crucial role in improving efficiency, resource allocation, decision making, and overall performance in a wide range of real-world problems. Here are some notable applications of optimization in our everyday lives:  
1. **Transportation and routing:** Optimization algorithms are used in transportation systems to optimize routes, schedules, and logistics. Whether it involves finding the shortest path for navigation apps, optimizing traffic signal timings, or planning public transportation routes, optimization helps minimize travel time, reduce congestion, and enhance overall transportation efficiency [21; 22; 23]. 2. **Resource management:** Optimization techniques are employed in diverse areas of resource management. For instance, energy companies optimize power generation and distribution to meet demand while minimizing costs. Water management systems optimize water distribution to ensure equitable supply and minimize wastage. Optimization is also applied in inventory management, supply-chain logistics, and workforce scheduling to optimize resource allocation and improve operational efficiency [24; 25; 26]. 3. **Financial planning and investment:** Optimization is widely used in financial planning and investment strategies. It helps investors optimize their portfolios by considering risk-return trade-offs. Optimization algorithms can determine the optimal allocation of funds across different assets or investment opportunities, aiming to maximize returns while managing risk within specified constraints [27; 28]. 4. **Production and manufacturing:** Optimization is crucial in production and manufacturing processes to improve efficiency, reduce costs, and maximize output. Production scheduling optimization algorithms help determine the optimal sequence and timing of manufacturing operations. Additionally, optimization is utilized in capacity plan ning, facility layout design, and supply-chain optimization to streamline operations and minimize waste [29, 30, 31]. 5. **Energy optimization:** Energy optimization plays a significant role in promoting sustainable practices and reducing environmental impact. Optimization techniques are employed in energy-efficient buildings, where they control heating, ventilation, and air-conditioning systems to optimize energy consumption while maintaining comfort levels. Smart grid technologies also leverage optimization algorithms to optimize power generation, distribution, and consumption, facilitating energy conservation [32, 33, 34, 35]. 6.