Learning to Solve an Order Fulfillment Problem in Milliseconds with Edge-Feature-Embedded Graph Attention

1 INTRODUCTION . The current decade has witnessed rapid development of online retailing , which generally has a very large operation scale and poses new challenges to supply chain management . One of the most important problems in the online retailing is the real-time order fulfillment decision , which aims to find the most cost-effective order-warehouse assignment after the customers place the orders online . The decisions are required to be made in real time , because the fulfillment system must complete hundreds of orders per second ( Acimovic & Graves , 2015 ; Xu et al. , 2009 ) . A good fulfillment policy could reduce expenses by millions of dollars for large online retailers such as Alibaba and Amazon . The order fulfillment problem in its most general form is as follows : There are a number of orders and a number of warehouses . Each order contains several items , which are at different inventory levels and have different sale-forbidden1 dates in these warehouses . Any warehouse can be assigned to serve any order , incurring delivery transportation cost and inventory management cost that vary depending on the locations of the orders and the types of items . An order can be split into multiple packages that are committed by different warehouses due to practical constraints , such as the shortage of inventory . A group of identical items in any order is defined as a suborder , which is the smallest assignment unit in this problem . The retailer seeks to find the optimal suborder-warehouse assignment policy that minimizes the overall cost in the order fulfillment . The order fulfillment problem falls into the category of generalized assignment problems , which is an NP-hard combinatorial optimization ( Kundakcioglu & Alizamir , 2009 ) ( An extra note on generalized assignment problem is listed in the Appendix A.1 ) . Traditional approaches to this problem can be divided into exact methods and heuristics . The exact methods guarantee finding optimal solutions by using a branch-and-bound search ( B & B ) with a mixed-integer programming formulation ( MIP ) , but they generally require a long time to solve and are even computationally intractable for large-scale cases . Because the order fulfillment is a real-time decision task in the online retailing , 1The sale-forbidden date is usually months ahead of the expiration date . Any item going beyond the saleforbidden date can no longer be sold . the exact methods are unable to meet the allowed maximum computation time ( tens of milliseconds ) . The heuristics are typically a set of specialized rules and are currently used in the industry because of the low computational cost ( Sethanan & Pitakaso , 2016 ; D ’ Ambrosio et al. , 2020 ) . As a trade-off , the optimality is sacrificed and once the conditions of the problem change , the rules must be re-designed carefully , which costs a lot of manpower . Recently , machine learning ( ML ) has made much progress on solving combinatorial optimization problems ( Bengio et al. , 2018 ) , e.g. , using supervised or reinforcement learning to solve the travelling salesman problem ( TSP ) and the vehicle routing problem ( VRP ) in an end-to-end approach ( Vinyals et al. , 2015 ; Kool & Welling , 2018 ; Lu et al. , 2020 ) . According to the research , ML methods can achieve stronger policies than heuristics without requiring specialized human knowledge . The computational cost of ML inference is extremely low . The ML methods are also self-adaptive to the changes on the problem conditions , because they are end-to-end methods from the original data to the solutions . This progress motivates us to explore whether order fulfillment in the supply chain industry can be addressed by ML methods . However , the nature of the order fulfillment problem is size-variant and contains heterogeneous information . Different scales of problem instances have varying numbers of orders , items and warehouses . How can we properly transform the combinatorial optimization problem into the input/output of an ML model ? How can we design an ML model that can effectively solve the varying scales of problem instances ? These questions pose great challenges to our work . In this paper , we will for the first time apply ML to solve the real-time order fulfillment decision problem by developing a graph mapping solution framework . A size-invariant graph model is developed to learn the hidden information on the heterogeneous graph . A novel graph attention mechanism is proposed to consider the high-dimensional edge features in the represented optimization problem . The specific contributions are threefold : ( 1 ) formulate the real-time order fulfillment decision problem with constraints of the modern online retailing and supply chain industry ; ( 2 ) propose a tripartite heterogeneous graph representation of the order fulfillment problem and develop an ML-based solution framework ; and ( 3 ) propose size-invariant and edge-feature-embedded graph attention to learn the hidden information in the heterogeneous graph . 2 RELATED WORK AND GAPS . Generally , applying ML on combinatorial optimization can be divided into two approaches . The first approach is to improve the branch-and-bound algorithm in the MIP formulation by using ML for variable branching ( Alvarez et al. , 2017 ) , node selection ( He et al. , 2014 ) and so on . Although these methods can guarantee exact solutions by relying on the branch-and-bound approach , the acceleration is limited . Our focus is on the other type of method , the end-to-end learning approach ( Bengio et al. , 2018 ) , which learns the optimization policy directly from the problem instances . The end-to-end ML models for combinatorial optimization require careful design , because both of the problem input and output vary in size ( Bengio et al. , 2018 ) . This circumstance means that the ML model must be able to address the data sizes that are completely unseen during training . To tackle this problem , the pointer network ( Ptr-Net ) is introduced based on the sequence-to-sequence model ( Vinyals et al. , 2015 ; Bello et al. , 2017 ) . The Ptr-Net directly uses an attention mechanism as a “ pointer ” to select a member of the input sequence as the output , regardless of the varying output dictionary sizes that depend on the problem instances . According to the experiments , the model performs well on the convex hull problem and TSP beyond the maximum sizes that they are trained on . Hu et al . ( 2017 ) ; Duan et al . ( 2019 ) further apply the Ptr-Net on the bin-packing problem to determine the optimal sequence in which the items are packed into bins . Apart from Ptr-Net , Kool & Welling ( 2018 ) solves the TSP based entirely on the attention mechanism similar to the Transformer architecture ( Vaswani et al. , 2017 ) . The input data is the ( x , y ) coordinates of all of the nodes , and the model calculates the attention between every pair of nodes . Finally , a sequence of nodes is generated as the optimal result . However , current studies based on Ptr-Net are mainly designed for sequential decisions , which might not fit the pattern for the studied assignment problem . According to Vinyals et al . ( 2016 ) ; Hu et al . ( 2017 ) , the use of the RNN as an encoder in Ptr-Net makes it difficult to represent long input sequences and suffers heavily from the order of the input data . The graph-base models , e.g. , the graph convolutional network ( GCN ) and graph attention network ( GAT ) ( Veličković et al. , 2018 ) , have also been applied in several studies , because many optimization problems can be represented as a graph . Dai et al . ( 2017 ) attempts to solve the minimum vertex cover problem and TSP by embedding the graph with a structure2vec network and iteratively adding the best node to construct the solution using reinforcement learning . Ma et al . ( 2019 ) proposes the graph Ptr-Net by using graph embedding as the input of Ptr-Net to solve the TSP . Li et al . ( 2018 ) first uses a GCN to generate a large number of candidate solutions to the combinatorial optimization . Then , a guided tree search is performed to refine the solution . However , most mentioned studies address homogeneous graphs without high-dimensional edge features , such as the TSP and VRP problems , where the input is a set of ( x , y ) coordinates . In the studied problem , the features of orders , items and warehouses are heterogeneous and the relationships between them are important , such as the tiered pricing rules between orders and warehouses , and the inventory between items and warehouses with different sale-forbidden dates . Current studies lack proper consideration of these important highdimensional edge features and the heterogeneous information between different types of nodes and edges . 3 PROBLEM STATEMENT . The mathematical formulation of the studied order fulfillment problem for modern online retailing is formulated in ( 1a ) - ( 1h ) . Table 1 lists the notation . For better understanding , a brief introduction on the generalized assignment problem is given in Appendix A.1 , which is the basics of the order fulfillment problem . min zkl ∑ k∈K ∑ i∈I ( C ( 0 ) ki xki + C ( 1 ) ki wki ) + ∑ k∈K ∑ j∈J ∑ t∈T Pj + kjt ( 1a ) s.t . zkl ≤ xki , ∀k ∈ K , ∀i ∈ I , ∀l ∈ L ( i , · ) , ( 1b ) ∑ k∈K zkl = 1 , ∀l ∈ L , ( 1c ) ∑ t∈T dklt = Olzkl , ∀k ∈ K , l ∈ L , ( 1d ) ∑ l∈L ( · , j ) dklt ≤ Skjt , ∀k ∈ K , ∀j ∈ J , ∀t ∈ T , ( 1e ) xkiD + wki ≥ ∑ l∈L ( i , · ) ( Wj ∑ t∈T dklt ) , ∀k ∈ K , ∀j ∈ J , ∀i ∈ I , ( 1f ) Fkjt + + kjt − − kjt = Skjt − ∑ l∈L ( · , j ) dklt , ∀k ∈ K , ∀j ∈ J , t = 0 , ( 1g ) Fkjt + + kjt − − kjt = Skjt − ∑ l∈L ( · , j ) dklt − −kjt−1 , ∀k ∈ K , ∀j ∈ J , t ≥ 1 ( 1h ) The objective of the optimization problem ( 1a ) is to minimize the overall fulfillment cost , including the delivery cost and the inventory management cost , by making the optimal assignment between warehouses and suborders . The delivery cost consists of the first weight cost and additional unit cost according to the tiered pricing rule in the logistics . The inventory management cost denotes the loss of the cargo value , which is the result of the items that become sale-forbidden . Constraint ( 1b ) denotes that if warehouse k serves any suborder in order i , then xki = 1 . Constraint ( 1c ) denotes that any suborder should be served by one and only one warehouse . Constraint ( 1d ) denotes that the sum of the selected items before different sale-forbidden periods in warehouse k must equal the gross item quantity of the suborder l. Constraint ( 1e ) means that any item j can not be out-of-stock . Constraint ( 1f ) represents the tiered pricing system in logistics , where the right-hand side of the equation is the total weight of the served part of order i by warehouse k. Constraints ( 1g ) - ( 1h ) represent the item picking sequence in the warehouse . Because any item going into sale-forbidden periods can no longer be sold , they must be picked from near to far . 1 Any suborder l ∈ L belongs to a unique order i ∈ I and has a group of identical item j ∈ J . 2 The package ( k , i ) denotes that part of or all the suborders in order i ∈ I are served by the warehouse k ∈ K and are delivered as one package . Note that zkl is the key decision variable , and it represents the assignment between warehouses and suborders . When we make decision , two constraints are the most important : ( 1 ) Each suborder should be assigned to only one warehouse , represented by ( 1c ) ; ( 2 ) The item inventory in the assigned warehouse must not be less than the demand of the suborder , represented by ( 1d ) - ( 1e ) . Other variables and constraints are auxiliary . For example , dklt , +kjt , − kjt are used mainly to represent the item picking sequence ; and xki , wki are used to represent the difference between the first weight and the additional weight in the tiered pricing rules .

Summary: A Graph Neural Network (GNN) model is designed to enable the supervised learning of optimal solutions for an order fulfillment problem in supply chain management. This is a problem that must be solved in real-time, making a GNN whose forward computations are quick an attractive option.  The proposed GNN model has three types of nodes corresponding to orders, items, and warehouses. Edges between the nodes of different types represent relationships between them (e.g., an item is in an order). Nodes and edges may have feature vectors representing additional information, e.g., the identity of a warehouse or certain costs on the edges. The GNN is designed to take these features into account.  The model is trained using optimal solutions as labels in a binary classification formulation. Predictions are post-processed to guarantee a feasible solution in the constraints of the order fulfillment problem.  Experimental results on randomly generated instances show that a GNN trained on small instances of this problem performs well even on larger instances, yielding near-optimal solutions in a short amount of time compared to simple GNN solutions, a non-ML heuristic, and an exact solver.