# Learning-Based Optimisation for Integrated Problems in
## 5 Discussion  
As evident from the reviewed State of the Art, the adoption of encoder-decoder networks in combinatorial problem solving has already resulted in significant advancements in the field of optimisation. The ability of advanced encoder networks (e.g., GNNs and GANs) to process a rich problem context and the fact that they are no longer limited by the fixed input size, suggests that more general frameworks for different combinatorial problems are on the horizon. In fact, the first attempts to develop universal end-to-end NCO frameworks for different types of optimisation problems are already underway (see [135; 136; 137]). Nevertheless, strategic system design problems, such as network planning, identification of the customer regions, and strategic mode choice, have not yet been extensively addressed by NCO research. Data-driven solutions, such as RL, have their roots in optimal control, and, as a result, the application of NCO on static, sparse networks appears to be over-sophisticated for the optimisation community. However, problems involving integrated system design and operational planning, such as the dynamic Location Routing Problem (LRP) (see, e.g., [138]), are well-suited candidates for learn-to-improve NCO or HRL. In the first case, simple optimisation heuristics could be used alongside the NCO model. At each step, the algorithm provides suitable terminal locations, while the NCO agent calculates transport costs for each partial solution (e.g., routes). For large LRP problems, replacing the optimisation heuristics with another, higher-level RL agent may be feasible. This would result in an HRL approach for an integrated system design problem. In fact, end-to-end strategies are more suitable for dynamic, online sequential problems (e.g., routing, sorting at terminals, scheduling, and resource assignment) because agent policy constructs decision sequences based on input within the same environment. In contrast, the hybrid learn-to-configure strategy indicates applicability in practical scenarios involving nested problems, such as vehicle fleet optimisation with an integrated assignment to trans-shipment terminals and vehicle routing, where complex sub-problems can be effectively encapsulated within an individual simulation environment. A learn-to-improve strategy is more suitable for strategic and static problems, or for problems requiring fast and precise solutions, where the solution search is guided by an exact method. Generating the initial solution with a simulator and applying NCO to accelerate local search yields more stable and reliable results. The combination of service network (e.g., determine modes, vehicles or service times) and operational planning problems (e.g., routing), together with terminal-related batching and sequencing (e.g., assign cranes to containers), can be approached using both end-to-end and learn-to-improve strategies with either a single central agent or a Multi-Agent NCO setup. Scheduling, dispatching, vehicle routing, and sorting represent typical sequential decision problems in service and operational planning. If the resources are not redistributed (i.e., optimisation focuses on a single terminal/train/vehicle), a central NCO agent would be a good design choice. If resources are redistributed and optimisation occurs across multiple locations or vehicle fleets simultaneously, MARL approaches such as Qmix [79] or HRL [99] with a central upper policy and decentral lower agents can be employed. However, designing decentralised MARL with central policy poses particular challenges. The State of the Art contains limited insights regarding the established ways to rebalance rewards from multiple agents in order to achieve reasonable interaction in a shared environment. Another interesting practical use-case is optimising transport plans for a service network modelled as a sequential decision process on graphs, where each node represents either a train station, trans-shipment terminal, or port [70].