Paper: GNN-based Probabilistic Supply and Inventory Predictions in Supply Chain Networks (ICLR-2024)

Reviewer comment (Writing):
What is a lane? Is it an edge on the graph? This notion is quite untraditional both in the supply chain literature and the graph learning literature.

Author response:
Thank you for your valuable feedback. We have made significant revisions to enhance the clarity and readability of our paper. In the revised version, 

- Added explicit descriptions of our main contributions

- Provided a clear description (Section 3) of the Problem Description.  

- Introduced our generalized method (Section 4) for GNN-based event quantity/timing delta predictions

- Presented our GSP (GNN-based Supply Prediction) models as an application of the generalized method for shipment event predictions in supply chain networks (Section 5)

- Addressed the clarity problems you highlighted.

- Consistently use the term “edge” instead of “lane”

Reviewer comment (Novelty):
There are no innovations in terms of neural network architectures.

Author response:
We are grateful for your time in reviewing our revised paper!  Hope our response positively impacts your support of accepting our paper.

We'd like to underscore that our GNN (Graph Attention Network - GAT) layers, as defined in Equation (5), are dedicated to learning embedding representations (graph node embeddings) to accomplish predictive learning tasks. Notably, our approach avoids intricate and manual feature engineering tasks. The loss function we have formulated is tailored for GNN to optimize embedding representations, minimizing prediction losses for both edge-level and node-level labels. Our subsequent analysis, employing the UMAP visualization technique on Graph Node Embeddings, reveals that the GNN effectively learns representations and discerns patterns. We hope that you will find this response relevant.

Reviewer comment (Novelty):
As I am also a researcher working on the intersection of machine learning and supply chain management, I do not see that this paper fits with the audience of this conference. Simply using a GAN to solve a specific dataset does not excite the audience of this community.

Author response:
3. ICLR Relevance

We think that our generalized method provides the solution to a very interesting “supervised representation learning” problem (ICLR’s one of main topics) in which the labels (ground truth) are not directly provided in the form of event quantity/timing (main prediction targets), but only obtainable in the form of edge-level and/or node-level aggregated quantities at specific temporal granularities. 
The ICLR website indicates that one of the relevant topics is “applications in audio, speech, robotics, neuroscience,  biology, or any other field.”  We believe that our research may play a pivotal role in charting the path for the integration of AI within the supply chain domain.

Reviewer comment (Writing):
I find the discussion in Section 3.2 intimidating with an overload of indices. To my understanding, the difficulty of this paper only lies in the construction of the target variables (partly due to the dynamism of the system). Section 3.2 and 3.3 are simply preparing these target values, so I think the authors should make this information simpler. So far, Section 3.2 and 3.3 look more like a "project report" to the industry partner. For the audience of this conference, why should we care about these calculations leading to equation (12) and (18)? In the end, neural networks are just functional approximations: we match input with output. Thus, even though the output can be difficult to compute, preparing the target variables is just data-processing. Unfortunately, it may not be exciting for the audience of the ICLR conference to learn about how the authors process the data.

Author response:
2. Inventory Prediction Process, Backprop through the Process

In response to your feedback, in our revised paper, Equation (15) for the generalized method introduces a pre-determined process model Z that takes edge-level predicted quantity vectors and predicts node-level aggregated quantity vectors.  This process model corresponds to the inventory prediction process in supply chain contexts.  We moved the details regarding the inventory prediction process to Appendix D.  

Still, we’d like to highlight that, due to limitations in the underlying supply chain processes and systems for tracking actual shipments against planned shipments, it is often not possible to establish a direct one-to-one correspondence between actual and planned shipments. Consequently, we lack precise information regarding the exact discrepancies between actual and planned shipments in terms of both shipment event quantity and timing. To address this issue, GSP model learns from the accessible labels (ground truth) in the form of the edge-level daily outgoing supply quantities and the node-level weekly inventory quantities.  

Hence, we contend that this model goes beyond a standard GAN/neural network designed to match inputs with outputs. The uniqueness lies in how the labels across the graph are provided, and the calculation of the loss function is executed. We compute cumulative supply prediction loss to encompass errors in both event quantity and timing simultaneously, predict (or reconstruct) inventory within the model, and employ backpropagation through the inventory prediction process model. This approach enables the model to leverage both edge-level and node-level aggregated labels at varying temporal granularities. Our approach can be also successfully employed for predictions in scenarios where events are intermittent, errors may accumulate, and graphically-aggregated labels exist at different temporal granularities.

Reviewer comment (Presentation):
The presentation and clarity of the paper are major concerns, making it almost impossible to follow the details and check the correctness of the introduced methods. The authors directly dive into the methodology without clearly stating the problem and combine the modeling with discussions of GAT.

Author response:
Thank you for your valuable feedback. We have made significant revisions to enhance the clarity and readability of our paper. In the revised version, 

- Provided a clear description (Section 3) of the Problem Description.  

- Clearly indicated the target variables for prediction, the labeled variables for model learning, and the input feature variables. 

- Introduced our generalized method (Section 4) for GNN-based event quantity/timing delta predictions

- Presented our GSP (GNN-based Supply Prediction) models as an application of the generalized method for shipment event predictions in supply chain networks (Section 5)

- Addressed the presentation problems you highlighted.

Reviewer comment (Experiments):
It is rarely possible to immediately distinguish which parameters are known before making the prediction and which are decision variables. There is a lack of clarity about what the inputs of the prediction are, the definition of planned shipments, and whether the plan of future supply is known before making predictions.

Author response:
Planned Shipments

Planned shipments mean the original scheduled (planned) shipment events in the enterprise’s planning system. The planned shipments frequently fail to be executed as originally devised. We use the planned shipment as prior information when predicting the deltas (= differences between the plan and the actual) in quantity/timing.

Reviewer comment (Writing):
The overwhelming number of parameters and notations used without sufficient explanations makes the work even tougher to follow. It is necessary to explicitly set up the problem and introduce all assumptions and formulation details of the supply chain model, especially since the formulation and model of the supply chain appear different from the traditional literature.

Author response:
Thank you for your valuable feedback. We have made significant revisions to enhance the clarity and readability of our paper. In the revised version, 

- Provided a clear description (Section 3) of the Problem Description.  

- Clearly indicated the target variables for prediction, the labeled variables for model learning, and the input feature variables. 

- Introduced our generalized method (Section 4) for GNN-based event quantity/timing delta predictions

- Presented our GSP (GNN-based Supply Prediction) models as an application of the generalized method for shipment event predictions in supply chain networks (Section 5)

- Addressed the presentation problems you highlighted.

Reviewer comment (Theory):
The paper may not make a sufficient realistic contribution to real supply chain planning problems. In practice, supply chains are decentralized and divided, without a centralized decision maker overseeing everything; instead, local decision makers handle smaller-scale problems and have well-controlled supplier relationships that lead to accurate local predictions. Centralized data collection and prediction can be infeasible and unnecessary.

Author response:
Usefulness for Real-World Supply Chain Systems

We respectfully disagree with your point of view that collecting data and making centralized predictions can be infeasible and unnecessary in real-world supply chain systems. Many organizations employ established S&OP processes for centralized predictions and planning across nodes and lanes in the supply chain network. Therefore, accurately predicting supply executions in relation to the original plan is crucial for organizations to make better informed corrections to their existing plans. We believe that our work may play a pivotal role in charting the path for the integration of AI within the supply chain domain.

Reviewer comment (Theory):
The authors claim contributions regarding the design of a loss function that is bi-objective and controlled by a weight hyperparameter. In the optimization community, such formulations are often avoided because they are hard to interpret and justify, especially when the two objectives are on different scales (one daily error and the other weekly cumulated errors). It is unclear how to quantify the impacts of the two objectives on subsequent planning, and which type of error will lead to higher operational costs.

Author response:
Loss Function Design

In designing our loss function, we provided flexibility for the model to utilize either the edge-level aggregated quantity labels (e.g., daily supply on edges) or the node-level aggregated quantity labels (e.g., weekly inventory on nodes), or a combination of both. While we acknowledge your observation, we believe that employing a weight hyperparameter to balance different loss components is a common practice, often used for regularization techniques. Additionally, optimizing the weight hyperparameter to achieve a balance between the two prediction objectives poses a similar challenge to tuning other model hyperparameters, as demonstrated in our experiment results.

Reviewer comment (Experiments):
The baseline methods used are obviously not strong enough, and comparing the proposed method to them does not make the proposed method stand out. It is unclear if these are the only methods available to predict ongoing supply.

Author response:
Baselines

In our paper, we have included two primary baselines—the error of the original plan and the Croston method's error. 

As highlighted in the Related Work section, our problem setting significantly diverges from those of traditional approaches. There are currently no open-source dataset and state-of-the-art method/code available for assessing supply prediction performance in topologically complex supply chain networks at the scale we addressed in this paper. We aim for our work to establish a foundational reference, encouraging others to focus on AI applications within the realm of supply chain networks.

Reviewer comment (Experiments):
It is unclear if the introduced method is robust to disruptions. Since robustness to disruptions such as SKU shutdowns or sudden high labor shortages (as occurred during the pandemic) is crucial for usefulness, there is doubt as to whether the method can quickly capture dynamics under such conditions and still perform well.

Author response:
Supply Chain Disruption, Robustness

Our work exhibits robustness in the face of supply chain disruptions, including sudden changes in network topology. This resilience stems from our models learning from a vast amount of topologically diverse structured data during the training. Even when certain nodes or lanes are removed from the network topology at the time of inference, the model continues to identify patterns in the new topology and makes good predictions, as observed previously.

Reviewer comment (Experiments):
The extreme specificity of the methodology. Supply chain is an extremely relevant problem; however, the proposed architecture seems to be extremely tailored for this one specific application. How generalizable are these methods beyond the supply chain application? It would be beneficial to see experiments on a more diverse set of problems.

Author response:
Our method is very generalizable for making event quantity/timing delta predictions in graph-structured problem contexts where there are planned events without a one-to-one mapping to actual events. This implies that there is no ground truth as labeled data for quantity/timing variables for model learning. Our approach relies on labeled data in the form of edge-level and/or node-level aggregated quantities at specific temporal granularities, such as daily or weekly periods. 

Our revised paper now clearly introduces the generalized method (Section 4) for GNN-based event quantity/timing delta predictions. Subsequently, in Section 5, we present our GSP (GNN-based Supply Prediction) models as an application of this generalized method.

Our method is versatile and can be applied to more general cases of event delta predictions in graph-structured settings, given some known prior or planned event information. For instance, it can be employed in any time series forecasting with intermittent event data where the prediction objective is centered around cumulative quantity prediction accuracy over a specific time horizon, provided some historically estimated prior information of events. 

In designing our loss function, we provided flexibility for the model to utilize either the edge-level aggregated quantity labels (e.g., daily supply on edges) or the node-level aggregated quantity labels (e.g., weekly inventory on nodes), or a combination of both.

Reviewer comment (Experiments):
The set of baselines is extremely limited. The authors should provide additional learning-based prediction models from literature or by implementing sensible alternative approaches to the problem.

Author response:
Thank you for your insightful suggestions regarding more interesting baselines. We value your input!

In our paper, we have included two primary baselines—the error of the original plan and the Croston method's error. 

As highlighted in the Related Work section, our problem setting significantly diverges from those of traditional approaches. There are currently no open-source dataset and state-of-the-art method/code available for assessing supply prediction performance in topologically complex supply chain networks at the scale we addressed in this paper. We aim for our work to establish a foundational reference, encouraging others to focus on AI applications within the realm of supply chain networks.

Reviewer comment (Reproducibility):
Is there any stabilization process carried out in the output layer such as multi-head attention?

Author response:
In our experiments we performed hyperparmeter tuning as described in Appendix B (Model Training Details). We experimented with 2 or 3 GAT convolution layers with 3 or 8 multi-head attentions using the validation dataset.  We determined 2 GAT convolution layers with 3 multi-head attentions that result in the minimum validation loss.

Reviewer comment (Reproducibility):
Is any sort of transformation or concatenation done on the output features?

Author response:
The transformation is done to the model outputs in the format as shown in Figure 8. All values are normalized using a min-max normalization, where the min value is normally zero.

Reviewer comment (Writing):
Typo error could have been avoided in Page 4 - footnote no. 3 - "dailiy basis" needs attention.

Author response:
We corrected typos in our revised paper.