<figure><img src="image_1.png" /></figure>

HAL
open science

# Effect of Informed Demand Lead Time Under Imperfect Advance Demand Information

Koichi Nakade, Tsubasa Seino

<figure><img src="image_2.png" /></figure>

## ► To cite this version:

Koichi Nakade, Tsubasa Seino. Effect of Informed Demand Lead Time Under Imperfect Advance Demand Information. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2021, Nantes, France. pp.464-474, 10.1007/978-3-030-85874-2_49. hal-04030358

HAL Id: hal-04030358

https://inria.hal.science/hal-04030358

Submitted on 15 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_4.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Effect of Informed Demand Lead Time under Imperfect Advance Demand Information

Koichi Nakade¹[0000-0002-0933-6614] and Tsubasa Seino¹[0000-0003-0414-6675]

¹ Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya 466-8555, Japan
nakade@nitech.ac.jp

**Abstract.** This paper considers two types of imperfect advance demand information. The advance demand information is given each day, and the actual arrival time is stochastic. In a known demand lead time model, the arrival date of actual demand, which is called demand lead time, is informed, and it is fixed. In unknown demand lead time model, the demand information does not include the demand lead time. In both cases, the actual demand may not occur with a given probability, and urgent demand may appear each day, which requires a product in the same day. The ordering, lost sale, holding and return costs are incurred. In this paper, these two cases are compared under the same situation except demand lead time information. To derive the optimal policy on order and return, each of the models is formulated as a Markov decision process. The optimal ordering policy in each day is derived, which minimizes the total expected cost over a finite horizon. The experimental results show that the total expected cost under the optimal policy in the case of the known lead time is smaller than that in the unknown lead time case, but the difference becomes small when the holding cost is small or the fraction of urgent demand is large.

**Keywords:** Advance Demand Information, Markov decision process, imperfect

## 1 Introduction

To control the inventory system appropriately, the demand information in advance is important. This is called advance demand information (ADI). Use of ADI leads to appropriate production and inventory control, and it decreases amounts of products in inventory and backlogs.

The advance information is desired perfect, which means the information includes the arrival date of the corresponding actual demand, and it is performed certainly. In literature, perfect advance demand information is assumed basically (see Gallego and Özer [1] and Karaesmen [2], Liberopoulos [3] for example). Zhang et al. [4] consider the effect of sharing information of capacity for a production and demand information in a supply chain with a manufacturer and a retailer.

On the other hand, the demand may be cancelled by some reasons. For example, the parts of some products are needed for the downstream manufacturer, but the amount of actual demand may decrease, compared with informed advance demand information, because the demand of final products in the downstream manufacturer is reduced due to cancel or additional order by customers. In this case, the demand information is called imperfect.
Recently, imperfect ADI has been included. In some case the amount of demand informed by ADI may be changed later. Liberopoulos and Koukoumialos [5] discuss the effect of use of imperfect ADI for a capacitated production and inventory system. Song and Zipkin [6] consider the use of imperfect ADI in inventory systems. Gao, Xu and Ball [7] study imperfect ADI in an assembly system, and Bernstein and Decroix [8] consider a multiproduct system with imperfect ADI in a single period.

In addition, the actual arrival date of demand may be unknown. In Topan et al. [9], the demand lead time, which is a time interval between ADI and the arrival date of the corresponding actual demand, is not informed, and some theoretical results of optimal policies are derived in such a model. They also assume that urgent demand which requires an item in the same day as it happens may occur, because sudden requirement may happen due to the big demand in the downstream manufacturer.

The certainty of demand lead time is desirable to control inventory. On the other hand, making the demand lead time information more precise may lead to more additional cost or less demand because some customers do not want to decide the lead time in advance. In this paper, two demand information models with imperfect advance demand information are considered. In one model, the arrival date of the actual demand is informed. In another model, which is the model of Topan et al. [9], the advance demand information does not include the demand lead time. In both models, demand information is imperfect, and the corresponding actual demand may not appear with a given probability. Each model is formulated as a Markov decision process, and the optimal order and return policy is derived. The difference of optimal policies is discussed through sensitivity analysis.

The organization of this paper is as follows. In section 2, the model with imperfect ADI is described. In section 3, each of the models with and without informed demand lead time is formulated as Markov decision process. Numerical results are shown in section 4, and we discuss the property of both models. Conclusion is given in section 5.

# 2 Model Description

## 2.1 Model

We consider an ordering model of products in a single stage single product inventory system during multiple periods. Products or parts are ordered to the supplier, and finished products are supplied to the inventory after fixed replenishment lead time. The return of items is also considered. That is, when the number of items in inventory is very large, items may be returned to the supplier. Each demand is satisfied by receiving one item from the inventory.

At the beginning of period $t$, the demand information happens, whose amount is denoted by $W_t$. This value is stochastic and follows $q_d = P(W_t = d)$ ($d = 0,1, ..., d_{max}$), where $d_{max}$ is the maximal value which $W_t$ may take. Each of $W_t$ demand information appears as an actual demand in the period $t + \tau$. The upper and lower bounds of $\tau$ are denoted by $\tau_u$ and $\tau_l$, respectively, and $\tau$ follows the probability
distribution $p_{\tau}, \tau \in \{\tau_l, \tau_{l+1}, \dots, \tau_u\}$, where $p = \sum_{\tau=\tau_l}^{\tau_u} p_{\tau} \le 1$. Thus, with probability $1-p$, the actual demand does not occur.

In a case that the arrival date of the actual demand is known, which we call the case of a known demand lead time, demand lead time $\tau \in \{\tau_l, \tau_{l+1}, \dots, \tau_u\}$ is informed with probability $\frac{p_{\tau}}{p}$ for each demand information.

In an unknown demand lead time model, when the demand information arrives in period $t$, the probability that the actual demand arrives in period $t + \tau_l$ is $p_{\tau_l}$. When the demand information arrives at time $t$ and the actual demand is not found at $t + \tau - 1$, $(\tau = \tau_l, \dots, \tau_u - 1)$, the probability that it appears in period $t + \tau$ is $\frac{p_{\tau}}{1 - \sum_{k=\tau_l}^{\tau-1} p_k}$. Thus, when the demand information arrives and the actual demand is not found before $t + \tau_u - 1$, the demand appears with probability $\frac{p_{\tau_u}}{1 - p + p_{\tau_u}}$ and it does not appear with probability $\frac{1-p}{1-p+p_{\tau_u}}$. This unknown demand lead time model is the same as shown in [9]. The resulting probabilities, with which the actual demand appears in each period and the actual demand does not appear, coincide with those in the known demand lead time model.

The number of external urgent order which does not relate with demand information is denoted by $D_t^s \in \{0,1,\dots,d_{smax}\}$, and it follows the probability $u_d = P(D_t^s = d)$, $d=0,1,\dots,d_{smax}$.

The expected amount of total demand $D_t$ is $E[D_t] = pE[W_t] + E[D_t^s]$. As following Topan et al. [9], the sensitivity of demand information is denoted by $q = \frac{pE[W_t]}{pE[W_t]+E[D_t^s]} = \frac{pE[W_t]}{E[D_t]}$.

When the actual demand arrives, if there is no item in inventory, the demand is lost. The replenishment lead time for each order is fixed as $L$. In period $t$, the number of items which are ordered in period $t - L + l$ and will reach the warehouse in period $t + l$ is denoted by $z_l$, $l \in \{0,1,\dots,L\}$.

The cost incurred in each period is as follows: the ordering cost for each item is $c(>0)$, the lost sale cost is $c_e(>0)$, the holding cost rate for each item in inventory is $h(\ge 0)$, and the return cost of items from inventory to the supplier is $c_r$. If $c_r$ is negative, when the item is returned to the supplier, the warehouse receives reward $-c_r$. In addition, it is assumed that $c + c_r \ge h \cdot L$. This implies that the sum of cost for ordering and returning an item is higher than keeping an item in $L$ periods, which eliminates the possibility that it is best that an order is made and one item is returned to the supplier at the same time.

The number of items in inventory at the beginning of period $t$ is denoted by $x_t$, and the number of items returned to the supplier is $y_t$. Here we assume that $x_t \ge y_t$.

The order of events during period $t$ is as follows:

1. The information of advance demand $W_t$ arrives.

2. The state of the system is observed, and the size of replenished items in period $t + L$, $z_L$, and the number of returned items $y_t$ are determined. The ordering and returning costs are incurred.

3. Items which are ordered in period $t - L$ arrive at the inventory. Its size is $z_0$.
4. The actual demand and the external urgent demand happen, and each of them is satisfied by one of items consisting of $x_t - y_t$ items in inventory and $z_{0,t}$ arriving items.

5. Inventory holding costs and lost sale costs are incurred.

In the following, subscript $t$ is sometimes omitted. The planning horizon of our model is $T + 1$. At the beginning of period $T + 1$, advance demand information arrives, and decision of ordering and returning items is not made in this period, and the remaining actual demand is satisfied (or lost) by the remaining items in inventory and the items which have been ordered. The objective is to derive an optimal order and return policy which minimizes the expected total cost over a planning horizon.
Table 1 shows a list of notations which appear in section 2.

<table><caption>**Table 1.** Notations</caption><tr><td colspan="2"><strong>Model Description</strong></td></tr><tr><td><i>W</i><sub>t</sub></td><td>: the amount of informed demand in period <i>t</i>,<br/><i>q</i><sub>d</sub> = <i>P</i>(<i>W</i><sub>t</sub> = <i>d</i>): the distribution of <i>W</i><sub>t</sub>, 0 ≤ <i>d</i> ≤ <i>d</i><sub>max</sub><br/><i>τ</i></td><td>: demand lead time, <i>τ</i> ∈ {<i>τ</i><sub>l</sub>, <i>τ</i><sub>l</sub> + 1, ..., <i>τ</i><sub>u</sub>},<br/><i>p</i></td><td>= &sum;<sub><i>τ</i>=<i>τ</i><sub>l</sub></sub><sup><i>τ</i><sub>u</sub></sup> <i>p</i><sub><i>τ</i></sub> ≤ 1,<br/><i>D</i><sub>t</sub><sup>s</sup></td><td>: the amount of external urgent demand in period <i>t</i>,<br/><i>u</i><sub>d</sub> = <i>P</i>(<i>D</i><sub>t</sub><sup>s</sup> = <i>d</i>) : the distribution of <i>D</i><sub>t</sub><sup>s</sup>, 0 ≤ <i>d</i> ≤ <i>d</i><sub>smax</sub>,<br/><i>D</i><sub>t</sub></td><td>: the total demand in period <i>t</i>,<br/><i>q</i></td><td>= <div style="display: inline-block; vertical-align: middle; text-align: center;"><div style="border-bottom: 1px solid black; padding: 0 5px;"><i>pE</i>[<i>W</i><sub>t</sub>]</div><div style="padding: 0 5px;"><i>pE</i>[<i>W</i><sub>t</sub>]+<i>E</i>[<i>D</i><sub>t</sub><sup>s</sup>]</div></div> = <div style="display: inline-block; vertical-align: middle; text-align: center;"><div style="border-bottom: 1px solid black; padding: 0 5px;"><i>pE</i>[<i>W</i><sub>t</sub>]</div><div style="padding: 0 5px;"><i>E</i>[<i>D</i><sub>t</sub>]</div></div>,<br/><i>L</i></td><td>: the demand lead time,<br/><i>z</i><sub>l</sub></td><td>: the amount of arriving items <i>l</i> periods later,<br/><i>c</i></td><td>: an ordering cost,<br/><i>c</i><sub>e</sub></td><td>: a lost sale cost,<br/><i>h</i></td><td>: a holding cost rate,<br/><i>c</i><sub>r</sub></td><td>: the return cost rate,<br/><i>x</i><sub>t</sub></td><td>: the number of items in inventory in the beginning of period <i>t</i>,<br/><i>y</i><sub>t</sub></td><td>: the number of returned items to a supplier in period <i>t</i>,<br/><i>T</i></td><td>: the planning horizon.</td><td colspan="2"><strong>Unknown lead time case</strong></td></tr><tr><td colspan="2"><strong>Known lead time case</strong></td></tr><tr><td colspan="2"><i>A</i><sub>τ</sub></td><td>: the amount of demand which is informed <i>τ</i> periods before, <i>τ</i> ∈ {0,1, ..., <i>τ</i><sub>u</sub>},<br/><i>a</i><sub>τ</sub></td><td>: the realization of <i>A</i><sub>τ</sub>, <i>τ</i> ∈ {0,1, ..., <i>τ</i><sub>u</sub>},<br/><i>R</i><sub>τ</sub></td><td>: the number of actual demand among <i>A</i><sub>τ</sub> informed demand, <i>τ</i> ∈ {0,1, ..., <i>τ</i><sub>u</sub>}.<br/><i>B</i><sub>τ</sub></td><td>: the number of orders whose demand will occur <i>τ</i> periods later, <i>τ</i> ∈ {0,1, ..., <i>τ</i><sub>u</sub>},<br/><i>b</i><sub>τ</sub></td><td>: the realization of <i>B</i><sub>τ</sub>, <i>τ</i> ∈ {0,1, ..., <i>τ</i><sub>u</sub>},<br/><i>p</i><sub>τ</sub><sup>d</sup></td><td>= <i>p</i><sub>τ</sub>/<i>p</i>: the probability that each demand will arrive <i>τ</i> periods later if it actually arrives, <i>τ</i> ∈ {<i>τ</i><sub>l</sub>, <i>τ</i><sub>l</sub> + 1, ..., <i>τ</i><sub>u</sub>},<br/><i>B̂</i><sub>τ</sub></td><td>: the number of demand who will occur <i>τ</i> periods later among informed demand <i>W</i><sub>t</sub>, <i>τ</i> ∈ {<i>τ</i><sub>l</sub>, <i>τ</i><sub>l</sub> + 1, ..., <i>τ</i><sub>u</sub>}.<br/><i>p</i><sub>τ</sub></td></tr></table>
## 2.2 Two types of demand information

We consider two types of demand information as discussed above. Here, we show additional notations in each model.

### (1) Unknown demand lead time

This model is the same as the model of Topan et al. [9], except the terminal cost. In [9], the cost in period $T + 1$ is set as zero, whereas in this paper the cost is determined by the remaining demand and items in inventory. Let $A_τ$ denote the number of demand which is informed $τ$ periods before and the corresponding actual demand has not arrived yet ($τ \in \{0, 1, ..., τ_u\}$). Among $A_τ$ demand, the number of the actual demand in a period is denoted by $R_τ$. For $τ \in \{τ_l, ..., τ_u\}$, when $A_τ = a_τ$, $R_τ$ follows a binomial distribution with parameters $(a_τ, \frac{p_τ}{1 - \sum_{k=τ_l}^{τ-1} p_k})$, which can be proved by the assumption of the demand lead time. For $τ \in \{0, 1, ..., τ_l - 1\}$, $R_τ = 0$. Then at the end of each period, the number of items in inventory becomes $(x + z_0 - y - \sum_{τ=τ_l}^{τ_u} R_τ - D^s)^+$, where $(x)^+ = \max(x, 0)$. Let $a = (a_0, a_1, ..., a_{τ_u})$.

When the value of $A_τ$ is $a_τ$ at the beginning of period $t$ ($τ = 1, 2, ..., τ_u$), $a_0$ becomes $W_t$, and after the actual demand arrives, the amount of demand information whose corresponding actual demand has not arrived is $a_τ - R_τ$ for $τ = 1, 2, ..., τ_u$. Thus, at the beginning of period $t + 1$, $A_{τ,t+1} = a_{τ-1} - R_{τ-1}$ for $τ = 1, 2, ..., τ_u$.

### (2) Known demand lead time

Let $B_τ$ denote the number of orders whose demand will occur $τ$ periods later. Its value is denoted by $b_τ$. Since the actual demand for each order happens with probability $p$, the amount of demand in this period, denoted by $R_b$, follows a binomial distribution with parameters $(b_0, p)$.

When the number of demand information arriving in period $t$ is $W_t$, the probability that each demand will arrive $τ$ periods later if it actually arrives is $p_τ^d = p_τ/p$ for $τ \in \{τ_l, τ_l + 1, ..., τ_u\}$. Thus, the sequence $(\bar{b}_{τ_l}, ..., \bar{b}_{τ_u})$, where $\bar{b}_τ$ denotes the value which is added to the current $b_τ$ for $τ = τ_l, ..., τ_u$, follows the multinomial distribution with parameters $W_t$ and $(p_τ^d; τ = τ_l, ..., τ_u)$.

When $b$ is $(b_0, ..., b_{τ_u})$ before arrivals of ADI in period $t$, the state $b' = (b'_0, ..., b'_{τ_u})$ before their arrivals in period $t + 1$ is as follows:

$b'_{τ_u} = 0, b'_{τ} = b_{τ+1} + \bar{b}_{τ+1}, τ \in \{τ_l - 1, ..., τ_u - 1\}, b'_{τ} = b_{τ+1}, τ \in \{0, ..., τ_l - 2\}$. (1) Note that the decision on ordering and returning items is made after advance demand information arrives in each period.

# 3 Optimal Equations

The problems described above can be formulated as Markov decision processes.

### (1) Unknown demand lead time

This is the same as the formulation of Topan et al. [9] except the terminal cost. We note that the state is given as $(a, z)$, where $z = (x, z_0, z_1, ..., z_{L-1})$ and $z_τ$ denotes the number of demands which will arrive $τ$ times later. The state space is

$$S = \{(a, z); a = (a_0, a_1, ..., a_{τ_u}) \in \{0, 1, ..., d_{max}\}^{τ_u+1}, z \in X \times V^L\},$$
where $X = \{0,1, ..., L(d_{max}(\tau_u - \tau_l + 1) + d_{smax})\}$ and $V = \{0,1, ..., Ld_{max}(\tau_u - \tau_l + 1)\}$. The action space for state $z$ depends only on $x$ and is given by $A_x = \{(z_L, y): z_L \in V, y \in X, z_L \cdot y = 0, y \le x\}$, $x \in X$.

The optimal equations are found in [9] for period $t = 0,1, ..., T$. The terminal cost for state $(a, z)$ in period $T + 1$ is given by

$$ \hat{f}_{T+1}(a, z) = E \left[ c_r \left( x + \sum_{l=0}^{L-1} z_l - \sum_{\tau=0}^{\tau_u} O_{\tau} \right)^+ + c_e \left( \sum_{\tau=0}^{\tau_u} O_{\tau} - \sum_{l=0}^{L-1} z_l - x \right)^+ \right] $$

Here $O_{\tau}$ is the number of actual demand among $a_{\tau}$ orders and follows a binomial distribution with $(a_{\tau}, \hat{p}_{\tau})$, where $\hat{p}_{\tau} = p$ for $\tau \le \tau_l$ and $\hat{p}_{\tau} = (p_{\tau} + p_{\tau+1} + \dots + p_{\tau_u}) / (1 - (p_{\tau_l} + p_{\tau_{l+1}} + \dots + p_{\tau_{l-1}}))$ for $\tau_l + 1 \le \tau \le \tau_u$.

## (2) Known demand lead time

The state is shown as $(b, z)$, where $b = (b_0, b_1, ..., b_{\tau_u})$ and $z = (x, z_0, z_1, ..., z_{L-1})$. The state space is $S = Y \times Z$ where

$$ \begin{aligned} Y &= \left\{ b = (b_0, b_1, ..., b_{\tau_u}): b \in M^{\tau_l+1} \times \prod_{\tau=\tau_l+1}^{\tau_u} M_{\tau} \right\}, \\ Z &= \{z = (x, z_0, z_1, ..., z_{L-1}): z \in X \times V^L\}, \\ M_{\tau} &= \{0,1,2, ..., d_{max}(\tau_u - \tau + 1)\}, \tau \in \{\tau_l, ..., \tau_u\}, \text{ and} \\ M &= \{0,1,2, ..., d_{max}(\tau_u - \tau_l + 1)\}. \end{aligned} $$

The action space $A_x$ is the same as the unknown demand lead time case and it is given by

$$ A_x = \{(z_L, y): z_L \in V, y \in X, z_L \cdot y = 0, y \le x\}, \quad x \in X. $$

When the state $(b, z)$ is given in period $t$, under the optimal policy, the total expected cost from period $t$ to $T + 1$, is denoted by $\bar{f}_t(b, z)$. The optimal equations are given by

$$ \bar{f}_t(b, z) = \min_{(z_L, y) \in A_x} \{j_t(b, z, z_L, y)\} \quad t = 1, 2, ..., T, \quad (2) $$

$$ j_t(b, z, z_L, y) = cz_L + c_r y + \bar{N}(b_0, x + z_0 - y) + E[\bar{f}_{t+1}((\bar{b} + \bar{B}'), (x + z_0 - y - R_b - D_t^s)^+, z_1, ..., z_L)], \quad (3) $$

where $\bar{b} = (b_1, ..., b_{\tau_u}, 0)$, and $\bar{B}' = (\bar{B}'_0, ..., \bar{B}'_{\tau_u})$, where $(\bar{B}'_0, \bar{B}'_1, ..., \bar{B}'_{\tau_{l-2}}) = (0,0,...,0)$, $(\bar{B}'_{\tau_{l-1}}, ..., \bar{B}'_{\tau_{u-1}})$ follows a multinomial distribution with $W_{t+1}$ and $(p_{\tau}^d; \tau = \tau_l, ..., \tau_u)$, and $\bar{B}'_{\tau_u} = 0$. In addition,

$$ \bar{N}(b_0, x + z_0 - y) = hE[(x + z_0 - y - R_b - D_t^s)^+ | b_0] + c_e E[(R_b + D_t^s - x - z_0 + y)^+ | b_0]. \quad (4) $$

The terminal cost is given by

$$ \bar{f}_{T+1}(b, z) = E \left[ c_r \left( x + \sum_{l=0}^{L-1} z_l - O \right)^+ + c_e \left( O - \sum_{l=0}^{L-1} z_l - x \right)^+ \right], \quad (5) $$

where $O$ follows a binomial distribution with parameters $(b_0 + b_1 + \dots + b_{\tau_u}, p)$.
# 4 Numerical Experiments

Here we discuss the properties of optimal policies for unknown and known demand lead time via numerical experiments. The program is coded with C language and compiled by Intel Compiler 16.0 on the computer with Intel(R) Core (TM) i7-8700 CPU 3.20GHz and 32GB RAM.

Through the experiments $T=10$, $\tau_l = 1$, $\tau_u = 3$ and $L=1$. $d_{max}=3$ and the amount of demand $W_t$ follows the truncated Poisson distribution as $P(W_t = k) = e^{-\lambda} \frac{\lambda^k}{k!}$, $k = 0,1, \dots, d_{max} - 1$ and $P(W_t = d_{max}) = 1 - \sum_{k=0}^{d_{max}-1} P(W_t = k)$. $P(D_t^s = 0) = 1 - \lambda'$, and $P(D_t^s = 1) = \lambda'$. The numbers of states are 28160 and 308000 in the cases of unknown and known demand lead time, respectively. In the following, we set the expected number of total actual demand in each period is 1, that is

$$E[D_t] = pE[W_t] + E[D_t^s] = 1, \quad (6)$$

and the sensitivity of demand is

$$q = \frac{pE[W_t]}{pE[W_t] + E[D_t^s]} = \frac{pE[W_t]}{E[D_t]} = pE[W_t] \quad (7)$$

and thus

$$E[W_t] = \sum_{k=0}^{d_{max}-1} ke^{-\lambda} \frac{\lambda^k}{k!} + d_{max} \left( 1 - \sum_{k=0}^{d_{max}-1} e^{-\lambda} \frac{\lambda^k}{k!} \right) = \frac{q}{p}. \quad (8)$$

where $\lambda' = E[D_t^s] = 1 - q$. The distribution of $\tau$ is $p_1 = p_2 = p_3 = p/3$. Here, when $p$ and $q$ are given, we determine $\lambda$ to satisfying $E[W_t] = q/p$. For example, when $p=q$, $\lambda = 1.0254398$, and when $p = 0.7$ and $q = 0.5$, $E[W_t] = 5/7$ and $\lambda = 0.7216941$.

Cost parameters are initially set as $c = 100$, $c_e = 300$, $c_r = 0$ and $h = 20$. Some cost parameters are changed to find their sensitivity under optimal policies. The optimal policies can be calculated by the well-known value iteration method (see Puterman [10]). For example, in the known lead time case, we first compute $\bar{f}_{T+1}(b, z)$ for all states $(b, z)$ by (5), and $\bar{f}_t(b, z)$ is calculated through (2) and (4), from $t = T, T - 1, \dots, 1$, sequentially. The pair $(z_L, y)$ which minimizes the right hand side of (2) is an optimal action for the given state $(b, z)$ in period $t$. In the unknown lead time case, the policy can be computed similarly.

In the following, initially there are no items in inventory and no demand information. When the demand information arrives at the beginning of period $t = 1$, the first decision on the sizes of order and return is made. Thus, the total expected cost in the unknown demand lead time case is given by

$$\hat{f} = \sum_{k=0}^{d_{max}} q_k \hat{f}_1(k, 0, 0, 0, 0),$$

where $\hat{f}_1(k, 0, 0, 0, 0)$ is the total expected cost under the optimal policy when the initial state is $(k, 0, 0, 0, 0)$. In the known demand lead time case it is given as

$$\bar{f} = \sum_{k=0}^{d_{max}} q_k \sum_{b_1=0}^{k} \sum_{b_2=0}^{k-b_1} \sum_{b_3=0}^{k-b_1-b_2} \frac{k!}{b_1! b_2! b_3!} \left(\frac{1}{3}\right)^k \bar{f}_1(0, b_1, b_2, b_3, 0, 0).$$
First, we assume that there is no urgent demand, and all informed demand require items, that is, $p = q = 1$.

Table 2 shows the optimal decision in periods 1 to 5 for several states in the unknown and known demand lead time cases.

We first consider the unknown lead time case. After new ADI arrivals in period 0, the state becomes $(a_0,0,0,0,0,0)$, where $a_0 = 0,1,2,3$. As this value increases the number of the optimal order also increases. In period 2 or later, for state (0,2,0,0,0,0) one new item is ordered, whereas for state (0,0,2,0,0,0), two new items are ordered. Since $p=1$, in state (0,2,0,0,0,0) two demand arrive in three days with probability 1. Thus, to reduce the holding cost, in this state only one item is ordered. For state (0,0,2,0,0,0), two demand arrive in two days with probability one. Thus, to satisfy these demand, two items must be ordered because an order lead time exists.

Here we discuss the known lead time case. In period 1, the number of orders is the same as $b_1$ for any possible initial state in $(0, b_1, b_2, b_3, 0, 0)$ after ADI is informed. Since order lead time is one, the order is made to satisfy $b_1$ demand in the next period to reduce holding costs. The similar decision is made until period 9.

Table 3 shows the total expected costs under optimal policies when one cost parameter varies. In this example, the option of returning items is selected only for few possible states over a planning horizon, and thus when $c_r$ is changed the change of the total expected cost is small for both known and unknown cases.

As the holding cost increases, the difference of the total optimal expected cost between these cases is greater. When there is no holding cost, the expected cost in the known demand lead time case is the same as that in the unknown lead time case. In fact, in the case of unknown lead time, orders are made to satisfy all possible informed demand because keeping items in inventory makes no additional cost. In addition, return of an item is not optimal for any state in periods 1 to 7.

Here we assume that the informed demand may require no item, or the sudden demand happens. We compute the optimal polices for all cases combining $p=0.5, 0.7, 0.9, 1.0$ with $q=0.5, 0.7, 0.9, 1.0$. Note that $E[D_t^s] = 1-q$, and the terminal cost does not include the urgent demand. Table 4 shows the expected total cost under optimal policy for various $p$ under a given $q$. In each $q$, as $p$ is smaller the total cost is greater because of the missing actual demand against ADI. When $q$ is small, the effect by the known demand lead time is small. On the other hand, when $q$ is large, the effect of the demand lead time is large when $p$ is 0.9. When $q$ is small, there are three types of uncertainty on demand: the variety of demand lead time, possibility of actual demand and the urgent demand. Since the effect of uncertainty due to the urgent demand is great, the effect of the demand lead time information is weakened.

# 5 Conclusion

In this paper, the known demand lead time case is compared with the unknown demand lead time case under imperfect demand information. They are formulated as Markov decision processes. Through the numerical experiments, we find that the known demand lead time makes the lower total expected cost, but the difference is small when
the holding cost is small, or the fraction of urgent demand is large. In the latter cases, the demand lead time information is not important.

The extensive experiments and theoretical approach to compare the cases of known and unknown demand lead time are needed. In addition, when the demand size and demand lead time is bigger the optimal policy cannot be computed in the normal method of Markov decision process, and thus the other approximate or heuristic approach will be necessary to compute the sub-optimal order and return policy. They are left for future research.

### Acknowledgments

This paper is partially supported by JSPS Grants-in-Aid for Scientific Research (C) Number 19K04904.

<table><caption>**Table 2.** Optimal Decision in Periods 1 to 5</caption><thead><tr><th colspan="4">(a) Unknown lead time case</th></tr><tr><th>(a<sub>0</sub>, a<sub>1</sub>, a<sub>2</sub>, a<sub>3</sub>, x<sub>0</sub>, z<sub>L</sub>)</th><th>(z<sub>0</sub>, y<sub>0</sub>)</th><th>(a<sub>0</sub>, a<sub>1</sub>, a<sub>2</sub>, a<sub>3</sub>, x<sub>0</sub>, z<sub>L</sub>)</th><th>(z<sub>0</sub>, y<sub>0</sub>)</th></tr></thead><tbody><tr><td>(0,0,0,0,0,0)</td><td>(0,0)</td><td>(0,0,0,1,0,0)</td><td>(0,0)</td></tr><tr><td>(1,0,0,0,0,0)</td><td>(1,0)</td><td>(0,2,0,0,0,0)</td><td>(1,0)</td></tr><tr><td>(2,0,0,0,0,0)</td><td>(2,0)</td><td>(0,0,2,0,0,0)</td><td>(2,0)</td></tr><tr><td>(3,0,0,0,0,0,0)</td><td>(2,0)</td><td>(0,0,0,2,0,0)</td><td>(0,0)</td></tr></tbody><thead><tr><th colspan="4">(b) Known lead time case</th></tr><tr><th>(b<sub>0</sub>, b<sub>1</sub>, b<sub>2</sub>, b<sub>3</sub>, x<sub>0</sub>, z<sub>L</sub>)</th><th>(z<sub>0</sub>, y<sub>0</sub>)</th><th>(b<sub>0</sub>, b<sub>1</sub>, b<sub>2</sub>, b<sub>3</sub>, x<sub>0</sub>, z<sub>L</sub>)</th><th>(z<sub>0</sub>, y<sub>0</sub>)</th></tr></thead><tbody><tr><td>(0,0,0,0,0,0)</td><td>(0,0)</td><td>(0,2,0,0,0,0)</td><td>(2,0)</td></tr><tr><td>(0,0,0,1,0,0)</td><td>(0,0)</td><td>(0,0,0,3,0,0)</td><td>(0,0)</td></tr><tr><td>(0,0,1,0,0,0)</td><td>(0,0)</td><td>(0,1,1,1,0,0)</td><td>(1,0)</td></tr><tr><td>(0,1,0,0,0,0,0)</td><td>(1,0)</td><td>(0,3,0,0,0,0)</td><td>(3,0)</td></tr></tbody></table>

<table><caption>**Table 3.** Optimal Total Costs</caption><thead><tr><th colspan="4">(a) Sensitivity of total cost in c<sub>r</sub></th></tr><tr><th></th><th>c<sub>r</sub> = -20</th><th>c<sub>r</sub> = 0</th><th>c<sub>r</sub> = 20</th><th>c<sub>r</sub> = 40</th></tr></thead><tbody><tr><th>f&#x0302;</th><td>1358.30</td><td>1365.62</td><td>1372.93</td><td>1380.24</td></tr><tr><th>f&#x0303;</th><td>1200.44</td><td>1207.61</td><td>1214.78</td><td>1221.95</td></tr></tbody><thead><tr><th colspan="4">(b) Sensitivity of total cost in h</th></tr><tr><th></th><th>h = 0</th><th>h = 20</th><th>h = 40</th></tr></thead><tbody><tr><th>f&#x0302;</th><td>1214.78</td><td>1372.93</td><td>1498.96</td></tr><tr><th>f&#x0303;</th><td>1214.78</td><td>1214.78</td><td>1214.78</td></tr></tbody><thead><tr><th colspan="4">(c) Sensitivity of total cost in c<sub>e</sub></th></tr><tr><th></th><th>c<sub>e</sub> = 200</th><th>c<sub>e</sub> = 300</th><th>c<sub>e</sub> = 400</th></tr></thead><tbody><tr><th>f&#x0302;</th><td>1322.15</td><td>1372.93</td><td>1413.69</td></tr><tr><th>f&#x0303;</th><td>1178.91</td><td>1214.78</td><td>1250.66</td></tr></tbody></table>
<table><caption>**Table 4.** Optimal Total Cost Under Different *p* and *q*</caption><thead><tr><th></th><th colspan="4" style="text-align:center;">(a) q=1.0</th></tr><tr><th></th><th style="text-align:center;">p = 0.5</th><th style="text-align:center;">p = 0.7</th><th style="text-align:center;">p = 0.9</th><th style="text-align:center;">p = 1.0</th></tr></thead><tbody><tr><th style="text-align:center;">f̂</th><td style="text-align:right;">1616.10</td><td style="text-align:right;">1541.19</td><td style="text-align:right;">1439.13</td><td style="text-align:right;">1372.93</td></tr><tr><th style="text-align:center;">f̅</th><td style="text-align:right;">1527.86</td><td style="text-align:right;">1403.33</td><td style="text-align:right;">1267.55</td><td style="text-align:right;">1214.78</td></tr></tbody><thead><tr><th></th><th colspan="4" style="text-align:center;">(b) q=0.9</th></tr><tr><th></th><th style="text-align:center;">p = 0.5</th><th style="text-align:center;">p = 0.7</th><th style="text-align:center;">p = 0.9</th><th style="text-align:center;">p = 1.0</th></tr></thead><tbody><tr><th style="text-align:center;">f̂</th><td style="text-align:right;">1635.06</td><td style="text-align:right;">1572.02</td><td style="text-align:right;">1496.88</td><td style="text-align:right;">1456.13</td></tr><tr><th style="text-align:center;">f̅</th><td style="text-align:right;">1567.22</td><td style="text-align:right;">1481.12</td><td style="text-align:right;">1417.10</td><td style="text-align:right;">1387.14</td></tr></tbody><thead><tr><th></th><th colspan="4" style="text-align:center;">(c) q=0.7</th></tr><tr><th></th><th style="text-align:center;">p = 0.5</th><th style="text-align:center;">p = 0.7</th><th style="text-align:center;">p = 0.9</th><th style="text-align:center;">p = 1.0</th></tr></thead><tbody><tr><th style="text-align:center;">f̂</th><td style="text-align:right;">1638.59</td><td style="text-align:right;">1589.11</td><td style="text-align:right;">1530.88</td><td style="text-align:right;">1498.84</td></tr><tr><th style="text-align:center;">f̅</th><td style="text-align:right;">1591.06</td><td style="text-align:right;">1529.36</td><td style="text-align:right;">1470.92</td><td style="text-align:right;">1450.93</td></tr></tbody><thead><tr><th></th><th colspan="4" style="text-align:center;">(d) q=0.5</th></tr><tr><th></th><th style="text-align:center;">p = 0.5</th><th style="text-align:center;">p = 0.7</th><th style="text-align:center;">p = 0.9</th><th style="text-align:center;">p = 1.0</th></tr></thead><tbody><tr><th style="text-align:center;">f̂</th><td style="text-align:right;">1603.83</td><td style="text-align:right;">1562.02</td><td style="text-align:right;">1512.67</td><td style="text-align:right;">1486.08</td></tr><tr><th style="text-align:center;">f̅</th><td style="text-align:right;">1563.71</td><td style="text-align:right;">1513.11</td><td style="text-align:right;">1453.87</td><td style="text-align:right;">1424.61</td></tr></tbody></table>

**References**

1. Gallego, G., Özer, Ö.: Integrating replenishment decisions with advance demand information. Management Science, 47(10), 1344-1360 (2001).

2. Karaesmen, F.: Value of advance demand information in production and inventory systems with shared resources. Handbook of Stochastic Models in Manufacturing System Operations, 139-165, Springer, New York (2013).

3. Liberopoulos, G.: On the tradeoff between optimal order base stock levels and demand lead-times, European Journal of Operational Research, 190(1), 136-155 (2008).

4. Zhang, M., Wu, K., Sun, C., Pan, E. Optimal decisions for a two-echelon supply chain with capacity and demand information. Advanced Engineering Informatics, 39, 248-258 (2019).

5. Liberopoulos, G., Koukoumialos, S.: On the effect of variability and uncertainty in advance demand information on the performance of a make-to-stock supplier, MIBES Transactions International Journal, 2(1), 95-114 (2008).

6. Song, J. S., Zipkin, P. H.: Newsvendor problems with sequentially revealed demand information. Naval Research Logistics, 59, 601-612(2012).

7. Gao, L., Xu, S.H., Ball, M.O.: Managing an available-to-promise assembly system with dynamic short-term pseudo-order forecast. Management Science, 58(4), 770-790 (2012).

8. Bernstein, F. Decroix, G. A.: Advance demand information in a multiproduct system. Manufacturing and Service Operations Management, 17(1), 52-65 (2015).

9. Topan, E., Tan, T., van Houtum, G-J., Dekker, R.: Using imperfect advance demand information in lost-sales inventory systems with the option of returning inventory, IISE Transactions, 50(1), 246-264 (2018).

10. Puterman, M. L.: Markov decision processes: discrete stochastic dynamic programming, Wiley (1994).