Learning to Solve an Order Fulfillment Problem in Milliseconds with Edge-Feature-Embedded Graph Attention

1 INTRODUCTION . The current decade has witnessed rapid development of online retailing , which generally has a very large operation scale and poses new challenges to supply chain management . One of the most important problems in the online retailing is the real-time order fulfillment decision , which aims to find the most cost-effective order-warehouse assignment after the customers place the orders online . The decisions are required to be made in real time , because the fulfillment system must complete hundreds of orders per second ( Acimovic & Graves , 2015 ; Xu et al. , 2009 ) . A good fulfillment policy could reduce expenses by millions of dollars for large online retailers such as Alibaba and Amazon . The order fulfillment problem in its most general form is as follows : There are a number of orders and a number of warehouses . Each order contains several items , which are at different inventory levels and have different sale-forbidden1 dates in these warehouses . Any warehouse can be assigned to serve any order , incurring delivery transportation cost and inventory management cost that vary depending on the locations of the orders and the types of items . An order can be split into multiple packages that are committed by different warehouses due to practical constraints , such as the shortage of inventory . A group of identical items in any order is defined as a suborder , which is the smallest assignment unit in this problem . The retailer seeks to find the optimal suborder-warehouse assignment policy that minimizes the overall cost in the order fulfillment . The order fulfillment problem falls into the category of generalized assignment problems , which is an NP-hard combinatorial optimization ( Kundakcioglu & Alizamir , 2009 ) ( An extra note on generalized assignment problem is listed in the Appendix A.1 ) . Traditional approaches to this problem can be divided into exact methods and heuristics . The exact methods guarantee finding optimal solutions by using a branch-and-bound search ( B & B ) with a mixed-integer programming formulation ( MIP ) , but they generally require a long time to solve and are even computationally intractable for large-scale cases . Because the order fulfillment is a real-time decision task in the online retailing , 1The sale-forbidden date is usually months ahead of the expiration date . Any item going beyond the saleforbidden date can no longer be sold . the exact methods are unable to meet the allowed maximum computation time ( tens of milliseconds ) . The heuristics are typically a set of specialized rules and are currently used in the industry because of the low computational cost ( Sethanan & Pitakaso , 2016 ; D ’ Ambrosio et al. , 2020 ) . As a trade-off , the optimality is sacrificed and once the conditions of the problem change , the rules must be re-designed carefully , which costs a lot of manpower . Recently , machine learning ( ML ) has made much progress on solving combinatorial optimization problems ( Bengio et al. , 2018 ) , e.g. , using supervised or reinforcement learning to solve the travelling salesman problem ( TSP ) and the vehicle routing problem ( VRP ) in an end-to-end approach ( Vinyals et al. , 2015 ; Kool & Welling , 2018 ; Lu et al. , 2020 ) . According to the research , ML methods can achieve stronger policies than heuristics without requiring specialized human knowledge . The computational cost of ML inference is extremely low . The ML methods are also self-adaptive to the changes on the problem conditions , because they are end-to-end methods from the original data to the solutions . This progress motivates us to explore whether order fulfillment in the supply chain industry can be addressed by ML methods . However , the nature of the order fulfillment problem is size-variant and contains heterogeneous information . Different scales of problem instances have varying numbers of orders , items and warehouses . How can we properly transform the combinatorial optimization problem into the input/output of an ML model ? How can we design an ML model that can effectively solve the varying scales of problem instances ? These questions pose great challenges to our work . In this paper , we will for the first time apply ML to solve the real-time order fulfillment decision problem by developing a graph mapping solution framework . A size-invariant graph model is developed to learn the hidden information on the heterogeneous graph . A novel graph attention mechanism is proposed to consider the high-dimensional edge features in the represented optimization problem . The specific contributions are threefold : ( 1 ) formulate the real-time order fulfillment decision problem with constraints of the modern online retailing and supply chain industry ; ( 2 ) propose a tripartite heterogeneous graph representation of the order fulfillment problem and develop an ML-based solution framework ; and ( 3 ) propose size-invariant and edge-feature-embedded graph attention to learn the hidden information in the heterogeneous graph . 2 RELATED WORK AND GAPS . Generally , applying ML on combinatorial optimization can be divided into two approaches . The first approach is to improve the branch-and-bound algorithm in the MIP formulation by using ML for variable branching ( Alvarez et al. , 2017 ) , node selection ( He et al. , 2014 ) and so on . Although these methods can guarantee exact solutions by relying on the branch-and-bound approach , the acceleration is limited . Our focus is on the other type of method , the end-to-end learning approach ( Bengio et al. , 2018 ) , which learns the optimization policy directly from the problem instances . The end-to-end ML models for combinatorial optimization require careful design , because both of the problem input and output vary in size ( Bengio et al. , 2018 ) . This circumstance means that the ML model must be able to address the data sizes that are completely unseen during training . To tackle this problem , the pointer network ( Ptr-Net ) is introduced based on the sequence-to-sequence model ( Vinyals et al. , 2015 ; Bello et al. , 2017 ) . The Ptr-Net directly uses an attention mechanism as a “ pointer ” to select a member of the input sequence as the output , regardless of the varying output dictionary sizes that depend on the problem instances . According to the experiments , the model performs well on the convex hull problem and TSP beyond the maximum sizes that they are trained on . Hu et al . ( 2017 ) ; Duan et al . ( 2019 ) further apply the Ptr-Net on the bin-packing problem to determine the optimal sequence in which the items are packed into bins . Apart from Ptr-Net , Kool & Welling ( 2018 ) solves the TSP based entirely on the attention mechanism similar to the Transformer architecture ( Vaswani et al. , 2017 ) . The input data is the ( x , y ) coordinates of all of the nodes , and the model calculates the attention between every pair of nodes . Finally , a sequence of nodes is generated as the optimal result . However , current studies based on Ptr-Net are mainly designed for sequential decisions , which might not fit the pattern for the studied assignment problem . According to Vinyals et al . ( 2016 ) ; Hu et al . ( 2017 ) , the use of the RNN as an encoder in Ptr-Net makes it difficult to represent long input sequences and suffers heavily from the order of the input data . The graph-base models , e.g. , the graph convolutional network ( GCN ) and graph attention network ( GAT ) ( Veličković et al. , 2018 ) , have also been applied in several studies , because many optimization problems can be represented as a graph . Dai et al . ( 2017 ) attempts to solve the minimum vertex cover problem and TSP by embedding the graph with a structure2vec network and iteratively adding the best node to construct the solution using reinforcement learning . Ma et al . ( 2019 ) proposes the graph Ptr-Net by using graph embedding as the input of Ptr-Net to solve the TSP . Li et al . ( 2018 ) first uses a GCN to generate a large number of candidate solutions to the combinatorial optimization . Then , a guided tree search is performed to refine the solution . However , most mentioned studies address homogeneous graphs without high-dimensional edge features , such as the TSP and VRP problems , where the input is a set of ( x , y ) coordinates . In the studied problem , the features of orders , items and warehouses are heterogeneous and the relationships between them are important , such as the tiered pricing rules between orders and warehouses , and the inventory between items and warehouses with different sale-forbidden dates . Current studies lack proper consideration of these important highdimensional edge features and the heterogeneous information between different types of nodes and edges . 3 PROBLEM STATEMENT . The mathematical formulation of the studied order fulfillment problem for modern online retailing is formulated in ( 1a ) - ( 1h ) . Table 1 lists the notation . For better understanding , a brief introduction on the generalized assignment problem is given in Appendix A.1 , which is the basics of the order fulfillment problem . min zkl ∑ k∈K ∑ i∈I ( C ( 0 ) ki xki + C ( 1 ) ki wki ) + ∑ k∈K ∑ j∈J ∑ t∈T Pj + kjt ( 1a ) s.t . zkl ≤ xki , ∀k ∈ K , ∀i ∈ I , ∀l ∈ L ( i , · ) , ( 1b ) ∑ k∈K zkl = 1 , ∀l ∈ L , ( 1c ) ∑ t∈T dklt = Olzkl , ∀k ∈ K , l ∈ L , ( 1d ) ∑ l∈L ( · , j ) dklt ≤ Skjt , ∀k ∈ K , ∀j ∈ J , ∀t ∈ T , ( 1e ) xkiD + wki ≥ ∑ l∈L ( i , · ) ( Wj ∑ t∈T dklt ) , ∀k ∈ K , ∀j ∈ J , ∀i ∈ I , ( 1f ) Fkjt + + kjt − − kjt = Skjt − ∑ l∈L ( · , j ) dklt , ∀k ∈ K , ∀j ∈ J , t = 0 , ( 1g ) Fkjt + + kjt − − kjt = Skjt − ∑ l∈L ( · , j ) dklt − −kjt−1 , ∀k ∈ K , ∀j ∈ J , t ≥ 1 ( 1h ) The objective of the optimization problem ( 1a ) is to minimize the overall fulfillment cost , including the delivery cost and the inventory management cost , by making the optimal assignment between warehouses and suborders . The delivery cost consists of the first weight cost and additional unit cost according to the tiered pricing rule in the logistics . The inventory management cost denotes the loss of the cargo value , which is the result of the items that become sale-forbidden . Constraint ( 1b ) denotes that if warehouse k serves any suborder in order i , then xki = 1 . Constraint ( 1c ) denotes that any suborder should be served by one and only one warehouse . Constraint ( 1d ) denotes that the sum of the selected items before different sale-forbidden periods in warehouse k must equal the gross item quantity of the suborder l. Constraint ( 1e ) means that any item j can not be out-of-stock . Constraint ( 1f ) represents the tiered pricing system in logistics , where the right-hand side of the equation is the total weight of the served part of order i by warehouse k. Constraints ( 1g ) - ( 1h ) represent the item picking sequence in the warehouse . Because any item going into sale-forbidden periods can no longer be sold , they must be picked from near to far . 1 Any suborder l ∈ L belongs to a unique order i ∈ I and has a group of identical item j ∈ J . 2 The package ( k , i ) denotes that part of or all the suborders in order i ∈ I are served by the warehouse k ∈ K and are delivered as one package . Note that zkl is the key decision variable , and it represents the assignment between warehouses and suborders . When we make decision , two constraints are the most important : ( 1 ) Each suborder should be assigned to only one warehouse , represented by ( 1c ) ; ( 2 ) The item inventory in the assigned warehouse must not be less than the demand of the suborder , represented by ( 1d ) - ( 1e ) . Other variables and constraints are auxiliary . For example , dklt , +kjt , − kjt are used mainly to represent the item picking sequence ; and xki , wki are used to represent the difference between the first weight and the additional weight in the tiered pricing rules .

Summary: Authors propose to solve an order fulfillment problem with imitation learning backed by GNNs. The specific version of the fulfillment problem is nontrivial: discussed has a hierarchy in order (an order can be decomposed into sub-orders), and items become forbidden over time. SCIP package is used as the expert policy, and authors employ behavior cloning. The specific model they use is a Graph Attention Transformer on the tripartite graph of orders, items, and warehouses. Compared to a reasonable heuristic, Pointer Network, and GAT which ignores edge features, the propose method yields much lower inference latency.