Periodic review and continuous ordering There exist many inventory control studies that consider either continuous review & continuous ordering, or periodic review & periodic ordering. Mixtures of the two are hardly ever studied. However, the model with periodic review and continuous ordering is highly relevant in practice, as information on the actual inventory level is not always up to date while making ordering decisions. This paper will therefore consider this case of periodic review and continuous ordering. Assuming zero fixed ordering costs, and allowing for a non-negative lead time and a general demand process, we first consider a one-period decision problem with no salvage cost for inventory remaining at the end of the period. In this setting we derive a base-line optimal order path, which is described by a simple newsvendor solution with safety stocks increasing towards the end of a review period. We then show that for the general, multi-period problem, the optimal policy in a period is to first arrive at this path by not ordering until the excess buffer stock from the previous review period is depleted, then follow the path for some time by continuous ordering, and stop ordering towards the end to limit excess stocks for the next review period. An important managerial insight is that, typically and possibly counterintuitively, no order should be placed at a review moment, although this may seem intuitive and is also the standard assumption in periodic review models. We illustrate for Normally distributed demand that adhering to the optimal ordering path instead can lead to cost reductions of 30% to 60% compared to periodic ordering, i.e. ordering exclusively at review moments. In the inventory control literature, the focus is often on two extreme cases: either periodic stock review and periodic ordering at that same review point, or continuous stock review and continuous order possibilities. See e.g. Axsäter (2006) and Silver et al. (1998) for discussions of such models. Mixtures of both extremes are hardly ever studied. For continuous review and periodic ordering this is not surprising, since in a singleitem setting the optimal policy will be equal to the pure periodic review solution with review periods equal to the time inbetween ordering points. Therefore, the sole contributions to the literature in this setting consider multi-item models Some work has also been done on the situation of continuous review and periodic ordering, in the specific case where multiple products are jointly replenished from the same supplier to achieve cost savings. Some of the first, concrete steps here were made by Goyal (1974) , who introduced an optimal algorithm for this problem. Since then a number of others have also studied this so-called "Joint Replenishment Problem". Recently, Roushdy et al. (2011) proposed an iterative method for a specific review structure and Zhang et al. (2012) studied this problem under correlated demands.. Interestingly and perhaps surprisingly, the other mixture, periodic review and continuous ordering, has never been studied to the best of our knowledge, at least not with "truly" continuous ordering. There have been a number of contributions where orders are allowed at a number of predefined times during a period. Two decades ago, Flynn and Garstka (1990) already formulated a model and according policies where orders are allowed to be placed at the start of sub-periods of equal length during a review period. Chiang (2001) proposes order splitting in a periodic review framework. That is, at the start of a period an order is placed, and this order arrives in batches with fixed interarrival times in the current period. This method provides a holding cost advantage, which is shown by minimizing costs under a service level constraint.. However, as mentioned before, none of the previous periodic review studies considers continuous ordering, i.e. potential ordering at any point in time, as we will do in this study. This will allow us to obtain new structural results and insights into periodic review inventory systems. Moreover, whereas models with a finite number of ordering opportunities typically have to be solved using time-consuming numerical techniques such as dynamic programming, our continuous formulation leads to simple newsvendor equations that determine the optimal ordering strategy during a review period. Interestingly, this strategy is also of a quite different nature than those proposed and studied before: it typically does not order at review moments.. As we are the first to explore this problem, we will assume a negligible fixed ordering cost. This allows us to study the maximum benefit of continuous over periodic ordering, and also to obtain insightful analytical results. We do so under quite general conditions of a non-negative lead time and a general continuous demand process. We remark that discrete demand processes can be analyzed in the same way, but the analysis and expressions are somewhat lengthier and do not provide additional insights. We therefore restrict the exposition to continuous demand.. In line with previous periodic review studies (including those discussed above), we assume that no (partial) inventory updates are done between reviews. Obviously, this is relevant for situations where substantial effort is required to receive such updates. Despite the current technological improvements that facilitate and automate stock counting, the assumption that inventories can be completely checked on a continuous base is often unrealistic. Raman et al. (2001) found evidence of inventory counting inaccuracy and product misplacement, Yano and Lee (1995) studied product quality issues, Nahmias (1982) analyzed spoilage due to product perishability, and Fleisch and Tellkamp (2005) performed a simulation study in which it was found that theft has severe consequences for the optimality of inventory policies that ignore resulting inaccuracies. Nevertheless, it is still worthwhile for future research to analyze whether partial information can be used to further lower costs, compared to not using that information at all, as is assumed in our initial exploration and more generally in the periodic review literature. We will return to this issue in the concluding section.. So, in our model, orders can be placed continuously and the quantity of interest is the order-up-to level for the inventory position at each time instant. We will derive the optimal policy in two phases. In the first phase, we assume that there is only one period and there is no salvage cost for inventory remaining at the end of the period. Given this simplifying assumption we formulate the total cost function and minimize it with respect to the order-up-to level at each time instant. The resulting policy will serve as the base-line for phase 2, where we consider the more realistic case with multiple periods in which remaining stock from any period remains present in the next period. We show that the optimal policy during a review period is to (i) not order until excess buffer stock remaining from the previous period is depleted, (ii) then apply continuous ordering following the base-line path for some time, but (iii) stop towards the end of the period in order to limit the excess buffer for the upcoming period.. The remainder of this paper is structured as follows. In Section 2 we derive the one-period base-line policy, and thereafter in Section 3 we adjust this policy to the general multi-period setting. In Section 4 we provide numerical examples and compare the policy to the periodic review, periodic ordering system, and in Section 5 we summarize our findings, discuss insights, and give concluding remarks.. 2 The one-period problem: base-line model Consider a single review period of length T > 0, for which at time 0 a stock level of 0 is observed. Stock information is updated only once per review period, i.e. at the start. However, non-negative orders can be placed at any time t ∈ [0, T ) and arrive after lead time L ≥ 0. Demand D r over a period of length r follows a distribution characterized by the continuous cdf F D r . Holding costs per unit per time unit are h > 0 and shortage costs per unit per time unit are p ≥ h. Fixed ordering costs are 0, and at time T we can freely dispose of remaining inventory. Please note that since information on demand (including theft, misplacement, etc.) is not made available between reviews, demand during a review period is not subtracted from the inventory position. That is, the inventory position at any time during a review period is defined as the starting inventory position plus all orders placed since the start of the current review period.. Any inventory strategy is characterized by the order-up-to level O t (0 ≤ t < T ) at any time t during a review period. Note that since demands during a review period are not subtracted from the inventory position, only strategies with non-decreasing order-up-to levels need to be considered. The aim is to find the values for O t that minimize the expected cost per period. An expression for that cost is obtained based on the following observation that holds for any t ∈ [0, T ): the inventory level at time t + L is equal to the inventory position at time t minus the demand in interval (0,t + L). Please note that we need to subtract demands in the interval (0,t + L) and not only in the interval (t,t + L), different from the standard analysis of continuous review inventory systems (see e.g. Axsäter (2006) , p. 90), since our definition of the inventory position at time t does not subtract the unknown demand in period (0,t).. So, the total expected cost per cycle is. where (x) + = max{0, x} and (x) − = max{0, −x}. Obviously, the best solution for the whole period is found by applying the optimal solution at any point during the period. Next, we therefore derive the optimal solution for a specific point in time during the period, after which we show that the point-for-point optimal solution indeed determines a feasible solution for the whole period as well.. For a specific value of t, the best value of O t is the one that minimizes. We easily get. and similarly. we obtain the first order condition for (1) as. So, the optimal order-up-to levelÕ t for a specific time t ∈ [0, T ), must satisfy. Please note thatÕ t is non-decreasing in t for any non-negative demand process D t+L . This implies that it is indeed feasible to achieve order-up-to levelÕ t during the whole review period (t ∈ [0, T )). Therefore, applying (2) for all t ∈ [0, T ) minimizes the expected cost over the whole review period.. A common assumption in both theory and practice is that demand over some time interval follows a Normal distribution. If we indeed assume a stationary Normal demand process with mean µ and standard deviation σ per time unit, i.e. D r ∼ N(µr, σ 2 r), then (2) gives. where Φ is the well-tabulated standard Normal distribution function.. We remark that the order-up-to levels in (3) may not always be non-decreasing over time for the (unrealistic) case that the holding cost rate is much smaller than the backorder cost rate, and the coefficient of variation σ /µ is large. However, rather than providing an interesting special case, this is an indication that the Normal distribution is unsuitable for estimating small quantiles of highly variable demand processes (due to the significant probability of demand being negative). Nevertheless, assuming Normal demand has shown to be suitable for many real-life situations, and we will also use it in our numerical investigation in Section 4.. Note from (3) that the existence of a positive lead time does not affect the nature of the problem. A larger lead time only implies that orders need to be placed earlier and that, correspondingly, a larger safety stock is needed at any time during a review period. Figure 1 illustrates the base-line path for the case with D r ∼ N(10r, 4r) (µ = 10, σ = 2), backorder costs p = 10, holding costs h = 1, lead time L = 0, and the review period normalized to unity. The figure shows how the order-up-to level increases at a (slightly) diminishing rate over time, as a combined effect in (3) of a linear increase in mean demand and a non-linear increase in variance over a period of length t + L. 3 The multi-period problem: general optimal policy. Consider the same set-up as in the previous section, but now under the more realistic assumption that there are multiple review periods and all inventory remaining at the end of a review period is carried over to the next review period. Any strategy has a maximum order-up-to-level O T . Since the base-line policy derived in the previous section (which is optimal in the one-period problem) does not take into account the extra expected holding costs that are incurred due to the safety stock that will be carried over to the next period, the optimal value of O T may be smaller thanÕ T . This can clearly only be achieved by deviating from the base-line path by not ordering-up-to more than a certain valueS. Indeed, note that the order-up-to levels in a review period [0, T ) determine the costs in "cycle" [L, T + L), but that the costs from T + L onward, i.e. for future cycles, are only affected by the maximum order-up-to levelS in a review period. Therefore, we will first derive the optimal policy for a given level ofS in some period (and for some value of the starting stock level S in that period), and then proceed to determine the unconstrained optimal policy.. So, let us first consider the best policy given a maximum order-up-to levelS ≤Õ T and starting stock level S ≤S, for some period. From the convexity of the cost function in (1) for any t ∈ [0, T ), it follows that costs in the interval [L, T + L) are minimized by staying as close to the base-line policy as possible at any time during the period [0, T ). It is easy to see that this is achieved as follows. First of all, order at the start of a period such that the pathÕ t is reached as soon as possible. If S ≤Õ 0 this is achieved immediately at time 0 by ordering up toÕ 0 at that time. If S >Õ 0 , then no orders are placed until the first time that a point a is reached wherẽ O a = S. This yields the starting inventory position S = max{S,Õ 0 }. Subsequently, orders at time t must be placed according toÕ t until time b whereÕ b =S, at which point ordering should be stopped and the next inventory review must be awaited.. So, given S andS such that S ≤S ≤Õ T , the optimal policy in a period is to set. where a solvesÕ a = S , and. Observe that if S ≤Õ 0 , then a = 0. An illustration of such a policy can be found in Figure 3 of Section 4. It is obvious from the above analysis (and Figure 3) that the optimal level,S, at which to stop ordering is independent of the starting stock level, S. The latter only affects the time it takes to reach the base-line path. This implies that the optimal policy is stationary in that it applies the same maximum order-up-to level for each period, independent of the starting stock level.. What remains is to find the optimal value forS. From the above discussion, it follows that the cost of the optimal policy for a period, given an initial inventory position S, is given by. This total cost function can be rewritten as. Using that: (i) the stock at the end of a period is equal toS minus the demand in that period, and (ii) one orders up toÕ 0 at the start of a period, we get the expected cost per period as. By minimizing this expected cost, the optimal valueS can be determined numerically for any demand process.. 4 Sensitivity analysis and comparison with pure periodic review

B I O B I O O O B O O O O O O O O B I O B I O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I


Optimal Pivot Selection Method Based on the Partition and the Pruning Effect for Metric Space Indexes Hisashi KURASAWA †a) , Daiji FUKAGAWA † †b) , Atsuhiro TAKASU † † †c) , and Jun ADACHI † † †d) , Members SUMMARY This paper proposes a new method to reduce the cost of nearest neighbor searches in metric spaces. Many similarity search indexes recursively divide a region into subregions by using pivots, and construct a tree-structured index. Most of recently developed indexes focus on pruning objects and do not pay much attention to the tree balancing. As a result, indexes having imbalanced tree-structure may be constructed and the search cost is degraded. We propose a similarity search index called the Partitioning Capacity (PC) Tree. It selects the optimal pivot in terms of the PC that quantifies the balance of the regions partitioned by a pivot as well as the estimated effectiveness of the search pruning by the pivot. As a result, PCTree reduces the search cost for various data distributions. We experimentally compared PCTree with four indexes using synthetic data and five real datasets. The experimental results shows that the PCTree successfully reduces the search cost. Similar object search is a key function for speeding up various algorithms. For example, image completion algorithms fill in the missing regions in images by searching for similar image regions from a large photo dataset [12] . Applications using such algorithms require an efficient similar object search algorithm to reduce the query execution cost.. Similarity search indexes are used for pruning objects dissimilar to a query [26] , and reduce the search cost, such as the distance computations and the disk accesses. Indexes that are based on a metric space can be applied to all types of data whose distances obey the metric space postulates such as the triangle inequality. Most indexing schemes use reference objects called pivots. They recursively divide a region into subregions by using pivots, and construct a tree structure index. That is, the methods of selecting pivots and dividing up the space by using these pivots determine the index structure and pruning performance. The existing pivot selection studies have mainly focused on increasing the number of pruned objects at a branch in the tree [6] .. However, most previous works do not take into account the balance of the tree. Tree balancing techniques are used in database management systems (DBMSs), and it is well known that the tree height reduction contributes to reduce the average search cost [5] . Thus, even if the pivots of the methods are enough good for pruning objects at the branch, the effect of search cost reduction may be limited according to the data distribution of a database. Therefore, we developed a new pivot selection method for optimizing both the pruning and the balancing [18] * . The main contributions of this paper are as follows.. • We propose a new information theoretic criterion called the Partitioning Capacity (PC) for pivot selection. The PC takes both the object pruning effect and index tree balance into account.. • We developed a metric space index called the Partitioning Capacity Tree (PCTree). The PCTree provides the necessary functions for constructing an index tree based on PC and for efficiently retrieving similar objects using the index tree.. • We show the efficiency of PCTree empirically through several experiments where we compared the PCTree with several metric space indexes using synthetic multi-dimensional data and five real datasets. Let M = (D, d) be a metric space defined for a domain of objects D and a distance function d : D × D → R. The Minkowski distance, Jaccard's coefficient, and the edit distance are examples of the distance function d. Our index deals with the space and these distances. The early indexing schemes of similarity searches used balanced tree structures. Vantage Point Tree (VPT) divides a region into two subregions based on the distance from a pivot [24] . Its distance is set for equally partitioning the objects in the region, so that the two subregions contain the same number of objects. Thus, VPT is a completely balanced binary tree. However, the early indexes are weak at pruning objects while searching. They selects pivots that are far away from the other objects and other pivots by using simple statical features, such as the variance.. Some indexing schemes focus on pruning dissimilar objects rather than balancing the tree. Some schemes use other partitioning techniques instead of Ball partitioning.. Vantage Point Forest (VPF) [25] and D-Index [11] divide a region into three subregions according to the distance from the pivot. They aim to exclude the middle area in the region because it is difficult to judge whether objects in this area are similar or dissimilar to a query. Generalized Hyper-plane Tree (GHT) [23] divides the space by using a hyper-plane equidistant from the two pivots.. Other schemes improve the pivot selection. Many studies have attempted to select pivots that are distant from the other objects and other pivots [6] , [24] . These methods used the variance [24] , the mean [7] , [11] , the sum [10] , and the ratio [22] . iDistance [13] selects pivots based on the clustering result and reduces the number of regions accessed during a search. OMNI-Family [15] chooses a set of pivots based on the minimum bounding region. MMMP [17] has a pivot selection based on the maximal margin, and it classifies dense regions. Moreover, some methods combine different pivot selection techniques [19] , [27] .. Still other schemes propose various index structures. Fixed Queries Array [8] is an array of distances between objects and pivots and it is used to save memory. List of Clusters (LC) [9] is a list of compact regions. It recursively divides the space into a compact region. iDistance consists of a list of pivots and a set of B + -trees for storing objects. Thus, its index structure is partially balanced. Although the B + -trees are balanced, the pivots are linearly accessed. Spatial Approximation Tree (SAT) [20] approximates a graph and uses pivots for approaching the query spatially. M-Tree [10] is a dynamic tree structure with object insertion and deletion. Slim-Tree [16] minimizes the overlap space managed by node in M-Tree.. We focus on the pivot selection method in this paper. The simple statistics-based approach [6] , [24] aims at determining the various distances between the pivot and each object and reducing the number of useless pivots. However, they don't consider the partitioning boundary of the pivot and the selected pivot may not be good for pruning. The clustering-based approach [13] , [17] aims at selecting a pivot and its partitioning boundary based on the clusters in the dataset. Although it can select better pivots for pruning than the simple statistics-based approach, it doesn't take into consideration the tree balancing. An imbalanced index tree increases the average search cost. Thus, we aim at reducing the search cost by taking both the object pruning and the tree balance into account. For this purpose, we extract even more information concerning the data distribution.

O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A game-theory analysis of charging stations selection by EV drivers Abstract-We address the problem of Electric Vehicle (EV) drivers' assistance through Intelligent Transportation System (ITS). Drivers of EVs that are low in battery may ask a navigation service for advice on which charging station to use and which route to take. A rational driver will follow the received advice, provided there is no better choice i.e., in game-theory terms, if such advice corresponds to a Nash-equilibrium strategy. Thus, we model the problem as a game: first we propose a congestion game, then a game with congestion-averse utilities, both admitting at least one pure-strategy Nash equilibrium. The former represents a practical scenario with a high level of realism, although at a high computational price. The latter neglects some features of the real-world scenario but it exhibits very low complexity, and is shown to provide results that, on average, differ by 16% from those obtained with the former approach. Furthermore, when drivers value the trip time most, the average per-EV performance yielded by the Nash equilibria and the one attained by solving a centralized optimization problem that minimizes the EV trip time differ by 15% at most. This is an important result, as minimizing this quantity implies reduced road traffic congestion and energy consumption, as well as higher user satisfaction. Any technology touted as environmentally-friendly is likely to have its place secured on news media around the globe. Among green solutions, Electric Vehicles (EVs), viewed by all as emission-free, clean and noiseless, are rapidly rising in popularity and expectations, also thanks to the predictable shortage of fossil fuel in the not-so-distant future. Indeed, EV mass-production and widespread adoption seem all but likely if some early hurdles are overcome, such as short driving range, lack of recharging infrastructure and long charging time.. Arguably, any road scenario in ten years' time will likely feature some ratio of EVs taking over the streets [1] . Oldfashioned gas pumps might also be gradually phased out by public charging stations, with electric outlets popping up in places such as curbside parking, parking lots and cab stands.. Even in this rosy scenario, one wonders when worries about vehicle range and availability of charging stations will be lifted and whether drivers will not be forced to plan their entire trip or commute around such availability, at least early on in charging station development. Finally, it is not clear when the "time consuming" tag will be removed from the task of car recharging.. Given the above concerns, ICT and Intelligent Transportation System (ITS) can step in and provide solutions that alleviate such concerns. Indeed, traditional navigation services could be integrated with the information provided by roadside network infrastructure and on-board user terminals through wireless communication [2] , [3] . A Central Controller (CC) could collect information on the vehicular traffic conditions and on the occupancy status of the charging stations through ITS facilities. Then, EV drivers with low battery level could send a request to the CC and ask for advice on the specific charging station to choose and the route to take.. The key point in this scenario, however, is that drivers that resort to such a navigation service will very likely behave as self-interested users, who aim at finding the best trade-off between the trip time (including the time they have to stop at the charging station) and the charge/change price they pay at the station. Thus, they will follow the advice provided by the CC only if they find it advantageous to themselves.. In this work, we show that the advice provided by the CC may not conform to the interests of EV drivers when it is obtained by solving a centralized optimization problem that maximizes the average per-EV utility. We demonstrate instead that the above requirement is satisfied when the problem is modeled as a non-cooperative game. Specifically, we resort to a congestion game [4] and a game with congestion-averse utilities [5] , where the players are the EVs with low battery level. EVs behave differently from ordinary players of ordinary games, in that they do not compute their strategy themselves, but rather follow the CC's advice. However, as explained below, the advice EVs received by the CC corresponds to the choice they would make themselves, had they all the necessary information.. In both congestion games and congestion-averse games, the decision to be made concerns the charging station that an EV should use, along with the route to take passing through such a station. The two game models exhibit a different level of realism and complexity; however, for both of them, we show that, when the CC uses the game solution to provide advice to the EVs, the following facts hold.. (i) The navigation strategies suggested by the CC correspond to Nash Equilibrium (NE) strategy profiles 1 , i.e., each EV finds the suggestion by the CC advantageous to itself and is willing to adhere to it.. (ii) When drivers value their trip time most, the advice provided by the CC leads to an average per-EV trip time that is very close to the minimum obtained by solving a centralized optimization problem, and much shorter than the one the drivers can obtain by adopting a greedy approach (e.g., always select the closest or the least congested charging station). This is highly desirable since, shortening the average per-EV trip time, contributes to reducing road congestion and energy consumption due to EVs.. The remainder of the paper is organized as follows. In Sec. II, we discuss previous work highlighting the novelty of our contribution. The system scenario is introduced in Sec. III, along with the statement of the problem under study. We motivate our work in Sec. IV, by showing that centralized optimal solutions may lead to advice that may not be followed by the EV drivers. The game-theoretic approach that we adopt for the problem solution can be found in Sec. V. In Sec. VI, we introduce the simulation scenario that we use to derive the results presented in Sec. VII. There, we show the low complexity of the proposed method and its excellent performance. The latter results are derived through the Simulation for Urban MObility (SUMO) tool [6] and using a real-world road topology. We draw our conclusions in Sec. VIII. We now show the performance that is attained through our approach, and compare it to the results obtained when a centralized optimization problem is solved at the CC as well as when a greedy selection of charging station and route is adopted. In order to derive the results in the cases where the CC generates its advice from the solution of the CAG or of the congestion game (labelled as CG in the plots), we proceed as follows. Every time interval T , the CC solves the game considering as players the EVs from which it has received a request. To do so, the CC starts from a random strategy profile, i.e., a random assignment of the facilities to the players. In the case of the congestion game, it assigns both the charging station and the corresponding route, while in the CAG, it assigns only the charging station and associates to each playercharging station pair the fastest route that takes the EV from its current road segment to the station, and from there to its destination. Player payoffs (i.e., trip times) are then computed through SUMO in the scenario described in Sec. VI. To derive the trip times, we assume that every non-player vehicle takes its originally-planned route, while players will conform to the advice of the CC, hence they will follow the suggested route.. Given the current strategy profile and player payoffs, the CC examines other strategies according to the solution algorithm in [5] for the CAG, and to the one in [31, Ch.7] for the congestion game. For every strategy, player payoffs are computed via SUMO as before. If a more profitable strategy is found for any of the players (i.e., if any of the players could deviate from the previous strategy), then the new strategy is adopted and the whole procedure is repeated until an NE is reached.. Unless otherwise specified, we consider that the CC takes the first NE it finds as the solution of the game. While subsequent equilibria could in principle be better, we found that in practical cases all the equilibria found for the CG yielded virtually the same payoffs. Using different starting solutions, e.g., assigning to each player the closest station or a station at random, did not influence the quality of the solution but only the convergence time, i.e., the number of iterations needed to find it.. For both the CAG and the congestion game, we evaluate the computational complexity, i.e., the number of strategies that the CC has to examine before reaching the game solution, which also corresponds to the number of SUMO runs. Then, we calculate the per-player trip time associated to such a solution. All results are averaged over 10 runs. We compare such values with the trip time obtained through the techniques described below. Optimal: the solution that the CC can obtain by minimizing the trip time averaged over all EVs that ask for advice. This solution in general is not an NE, thus it may not be followed by rational drivers. Greedy: the CC only disseminates information on the road travel time, and on the occupancy and the charging time at stations. Based on this knowledge, each EV independently makes its own decision by selecting the charging station and the route that are deemed to minimize its own trip time. Note that, in this case, the CC just informs the EVs without providing any advice, and the EV decision is taken disregarding the presence of other vehicles looking for a charging station. Now, let us initially neglect the presence of non-rational drivers and of EVs with low battery whose drivers do not use the navigation service. Fig. 5 depicts the number of strategies that the CC has to examine before the solution to the game is found, for both the CAG and the congestion game (CG). We stress that the CC returns its advice to EV drivers only once the game solution (which is a pure-strategy NE) has been reached, thus the computational burden is solely carried by the CC. The two plots in the figure refer to the cases where the average number of EVs that are low in battery and ask for advice (i.e., players) is, respectively, 20% and 60% of the average total number of vehicles simultaneously present in the road topology.. As expected, the complexity of the congestion game is always higher than that of the CAG and, in both cases, it increases as the number of players grows. In particular, for our range of player numbers, the CC always examines less than 4000 strategies before finding the solution in the case of the CAG, and less than 8000 in the case of the congestion game. We remark that one SUMO run only takes a few seconds, hence simulation time is manageable.. The plots also provide a striking comparison between the CAG and the congestion game. While the complexity of the former remains remarkably low, the complexity of the latter increases severely as the number of players grows beyond 60. On the contrary, the total number of EVs in the road topology has just a marginal impact on both the CAG and the congestion game solution time. These results indicate that the CAG model is highly scalable, hence it can be successfully applied even to very large, crowded system scenarios.. Next, one may wonder whether the solution obtained through the CAG is as good as the one of the congestion game, or if the gain in complexity we have with the CAG takes a high toll in terms of system performance. To answer this question, in Fig. 6 we show the average vehicle trip time, for both player and non-electric vehicles, again as the number of players is 20% and 60% of the total number of vehicles. The performance corresponding to the solutions of the two games are also compared to that of the optimal solution.. The figure shows that the average trip times of player and non-electric vehicles have the same qualitative behavior, with the former clearly being higher than the latter since players stop at a charging station during their trip. Also, comparing the two plots, it can be seen that the smaller the total number of vehicles simultaneously present in the road topology, the lower the traffic intensity and the shorter the average per-EV trip time. As for the comparison among the CAG, the congestion game and the optimal, the difference in performance can be barely noticed when the players are 20% of the total number of EVs (left plot of Fig. 6 ). When the percentage of players is large (right plot), the difference with respect to the optimal is limited in the case of the CAG, and it is again unnoticeable for the congestion game. This indicates that neglecting the contribution of player EVs to the travel time makes the CAG model less precise only when players represent the majority of vehicles on the road topology. Fig. 7 confirms such an observation. The figure highlights the different contributions to the average per-player trip time, due to the waiting time at the charging station, the service time (which is constant) and the travel time. The results refer to the CAG (top plot) and to the congestion game (bottom plot), when the players are 60% of all vehicles. It can be seen that the difference between the two game models mainly resides in the travel time, which is higher when the CAG solution is adopted. Fig. 8 depicts the 10th (dashed line) and the 90th (solid line) percentiles of the per-player trip time, when players are 20% (top) and 60% (bottom) of all vehicles. In the case of the 10th percentile, the difference, among the solution of the CAG, that of the congestion game and the optimal, can be barely detected. As for the 90th percentile, it can be observed that, when the optimal solution is adopted, a fraction of player EVs may experience a significantly longer trip time than under the congestion game or the CAG. This suggests that applying the optimal solution may lead to higher unfairness in the user performance. Intuitively, this lack of fairness is connected to the fact that some users will deviate from the optimal solution, which is therefore not an equilibrium.. We now investigate the benefit of our approach with respect to the aforementioned greedy scheme. Recall that the greedy technique assumes the EVs to have periodically updated information about road traffic and status of the charging stations. In spite of this, Fig. 9 clearly shows that a greedy approach cannot cope with the other techniques in terms of performance: the degradation that is observed is indeed severe and becomes exceedingly high as the number of players increases. Intuitively, this is due to many users selecting the (currently) least crowded station, which suddenly becomes overloaded (as in the well-known route-flapping effect). Fig. 9 also depicts the performance of the CAG when the CC does not solve the game using the first NE that is reached, but the NE that minimizes the average per-player trip time among the first 10 it finds. In the plots, we label this curve by CAG-10. Interestingly, such a simple enhancement to the solution procedure makes the CAG approach as effective as the congestion game and the optimal, without impairing its scalability.. In conclusion, not only modeling the system through a CAG is a feasible, practical approach to the problem, but its solution also leads to a performance that is remarkably close to the optimum and much better than that attained with a greedy scheme.. We now consider the case where not all EVs with low battery resort to the navigation service, rather they act according to the greedy scheme. Recall that this case also represents the behavior of non-rational EVs, i.e., EV drivers that ask the CC for advice but they do not follow its suggestion. The results portrayed in Fig. 10 refer to the case where there is an equal number of rational EVs (i.e., player EVs) that ask for advice and of EVs that instead do not resort to the CC. The plot shows that EVs that do not exploit the navigation service, on average, experience a higher trip time than those that use it. Such a difference in performance is particularly evident as the number of EVs with low battery increases. This further confirms that our game-theoretic approach always leads to solutions (i.e., advice from CC) that are advantageous to the EVs, thus increasing the user satisfaction.. Next, we assume that the information the CC can acquire through the ITS is not fully accurate. Specifically, Fig. 11 shows the effect of such inaccuracy when the waiting time at the charging stations is affected by a random jitter, uniformly distributed between 0 and 300 s. From the plots, we can see that the average per-EV trip time increases, and that a longer time is spent waiting at the stations. Indeed, stations that were free might have been advertized as busy and vice versa. As a consequence, EVs avoid free stations that are advertized as busy, while they flock to busy stations advertized as free.. In Fig. 12 , we consider that the number of charged batteries available at the charging stations is limited, i.e., EVs may have to wait until their own battery is charged. Fig. 12 refers to the case where at every time period T , the number of charged batteries available at each station is equal to twice the number of servers at the station. We observe a sharp increase in the average per-EV trip time as the number of player EVs increases. More interestingly, the right plot in Fig. 12 shows that only a fraction of such an increase is due to vehicles waiting for their battery to be charged. Rather, vehicles find it advantageous to travel very far away to find a station with charged batteries available -hence the longer travel timesand possibly waiting in line at such stations -hence the longer waiting times.. Finally, we focus on the CAG and look at the case in which stations have significantly different charge prices. More specifically, such prices are uniformly distributed in [10, 50] and K i = K. In Fig. 13 , we study the effect of the "value of time", 1/K, on the average trip time of player EVs. Recall that the higher 1/K, the more EV drivers value time with respect to money. As expected, we observe that if vehicles give more importance to the charge price rather than to the trip time, the latter tends to increase. Indeed, vehicles will be more willing to wait in line at the cheapest stations, as well as to make longer trips to reach them.. Tab. III highlights another interesting aspect: giving a higher importance to money rather than to time significantly decreases the computational complexity of finding an equilibrium. This effect has the following intuitive explanation: trip times depend on other players' choices, while charge price only depend on the station selected by each player. Therefore, the more important price is for a player, the less likely it is that her choice will be affected by the decisions of other players, which only influence trip times. Fig. 14 shows the average per-charge prices paid by the vehicles. When vehicles only consider time (1/K = ∞), prices are very close to the average. If price is also accounted for, we observe that (i) prices are lower, and (ii) fewer players (hence shorter trips) are associated to cheaper prices. Recently, both the academic and industrial communities have devoted a great deal of interest to EVs and to the use of ITS services in support of EV drivers. As an example, in [7] Ferreira et al. consider the case where the behavior of EV drivers, i.e., whether they drive to the closest or the cheapest charging station, depends on their profile (age or gender). The authors design a system that, through various communication technologies, provides EV drivers with several pieces of information, among which, the locations of charging stations. The burden of selecting the charging station, however, is left to the drivers, as the study of the trip time associated to different alternatives is not within the scope of [7] .. An analytical model for the study of the EVs trip time is presented in [8] . The road topology is modeled as a graph whose edges are associated with a fixed, i.e., non trafficdependent, waiting time. Charging stations are likened to multi-server queues, and a theoretical lower bound to the charging time is derived. The model, however, does not include the availability of a central controller and, unlike our study, it does not consider that vehicles may deviate from their originally-planned route in order to reach a suitable charging station. Thus, the study in [8] does not account for the EV travel time to and from a charging station. The presence of a central controller is considered in [9] , [10] , where the route of an EV is minimized while accounting for stop-overs at charging stations. A multi-objective decision-making model is also presented in [11] , where the gas station selection depends on drivers' personalized requirements and gasoline price, and it aims at minimizing travel distance and refuelling price. In these works, however, individual EV routing and charging are optimized through standard techniques, and the effect of such decisions on each other is not taken into account.. The works in [12] - [20] are mostly concerned with the EV consumption and its impact on the power grid. Indeed, what is a charging station from the viewpoint of EVs is at the same time an energy storage station from the viewpoint of the power grid -a place where surplus energy, produced during low-demand periods, can be stored for later usage.. In particular, in [14] the authors envision a central controller that predicts the EVs mobility and advises each EV about which charging station to use and when, so as to smooth the power consumption peak. The work in [14] , however, accounts neither for the time that EVs may have to wait in line at the charging station, nor for the fact that EVs may act strategically. A fully-distributed mechanism is proposed in [16] , which lets EVs select fast charging stations along a highway. The mechanism is based on a multi-agent approach and requires EVs to continuously interact in order to adapt to each others' individual decisions. The work in [17] proposes a family of algorithms that, by regulating the voltage fed to EVs using different charging stations, aim at minimizing the load factor, the load variance or the power losses over the grid. Similarly, the goal of [18] is to ensure that EVs can obtain the energy they need to recharge their batteries, without impairing the stability of the power grid. The work in [18] takes into account the behavior of EV drivers, but it aims at influencing it by means of monetary incentives. The study in [19] jointly addresses the optimal power flow and the EV charging problems. The authors show that the optimal power flow problem is generally non-convex and non-smooth, but it can be solved optimally using its convex dual problem for most practical power networks. In [20] , the rate at which EVs charge is controlled so as to lead to a better utilization of the power grid. A rather different approach is followed in [21] , in which vehicles are assumed to negotiate day-ahead charging schedules. The overall objective is to shift the load due to EVs to fill the overnight electricity demand valley. On a similar note, the work [22] looks at charging stations from the viewpoint of the power grid, viewing them as energy storage stations. The authors envision generating more energy when the demand is low, and storing it -under the guise of charged EV batteries -for usage during subsequent, highdemand periods. We remark, however, that the study of the impact of EVs on the power grid, although interesting, is not within the scope of our work. Indeed, properly accounting for such aspects as the integration of distributed power sources in the power grid, would require a totally different study [23] .. The study in [24] focuses on estimating the battery discharge time. The trips of the EVs are modeled using real data and traffic statistics, and vehicles are assumed to use the closest available charging station. Again, the waiting time at the charging station and the fact that EV drivers may significantly deviate from their planned route to reach a station are neglected.. A game-theoretic approach is adopted in [25] , whose main contribution is to provide an analytical framework that is suitable for capturing the interactions between charging stations and EVs. The latter are assumed to act in groups, and need to decide on their charging profiles. The problem is modeled as a generalized non-cooperative Stackelberg game, in which the charging stations act as leaders and the EV groups are the followers. With respect to this work, we account for the fact that EVs pursue a trade-off between charge price and trip time, and that such a trade-off can be vehicle-specific.. At last, we mention that in [26] , we presented a preliminary work that investigates which information is important that EV drivers receive through ITS. In particular, we showed the benefit of transmitting specific suggestions to EV drivers on which charging station to use with respect to the case where only mere updates on traffic conditions and charging station occupancy are provided. The evident advantages brought by specific advice motivated our present work, which is concerned about how such advice should be determined. A sketch of this work with a few preliminary results has been included in [27] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An

iteration

normalization

and

test

method

for

differential

expression

analysis

of

RNA-seq

data

Background:

Next

generation

sequencing

technologies

are

powerful

new

tools

for

investigating

a

wide

range

of

biological

and

medical

questions.

Statistical

and

computational

methods

are

key

to

analyzing

massive

and

complex

sequencing

data.

In

order

to

derive

gene

expression

measures

and

compare

these

measures

across

samples

or

libraries,

we

first

need

to

normalize

read

counts

to

adjust

for

varying

sample

sequencing

depths

and

other

potentially

technical

effects.

Results:

In

this

paper,

we

develop

a

normalization

method

based

on

iterating

median

of

M-values

(IMM)

for

detecting

the

differentially

expressed

(DE)

genes.

Compared

to

a

previous

approach

TMM,

the

IMM

method

improves

the

accuracy

of

DE

detection.

Simulation

studies

show

that

the

IMM

method

outperforms

other

methods

for

the

sample

normalization.

We

also

look

into

the

real

data

and

find

that

the

genes

detected

by

IMM

but

not

by

TMM

are

much

more

accurate

than

the

genes

detected

by

TMM

but

not

by

IMM.

What's

more,

we

discovered

that

gene

is

highly

associated

with

kidney

cancer

and

so

on.

Deep

DNA

sequencing

methods

(ChIP-seq

and

RNA-seq)

offer

distinct

advantages

in

increased

specificity,

sensitivity

and

genome-wide

comprehensiveness

that

are

leading

to

their

wider

use.

It

has

been

showed

that

splicing

variants

and

single

nucleotide

polymorphisms

can

be

detected

through

sequencing

the

transcriptome,

opening

up

the

opportunity

to

interrogate

allele-specific

expression

and

RNA

editing..

The

reads

produced

by

RNA-Seq

are

first

mapped

to

the

reference

genome

using

computer

programs.

Then,

the

output

of

RNA-Seq

can

be

summarized

by

a

sequence

of

'counts'

That

is,

for

each

gene,

it

gives

a

count

standing

for

the

number

of

reads

whose

mapping

starts

at

that

gene.

As

different

libraries

have

different

total

read

counts,

i.e.,

sequencing

depths.

In

order

to

compare

the

genes

expression

and

detect

distinction

between

libraries,

we

should

normalize

the

libraries.

The

aim

of

normalization

is

to

remove

systematic

technical

effects

that

occur

in

the

data,

and

ensure

that

technical

bias

has

minimal

impact

on

the

results.

Experience

with

microarray

data

showed

that

normalization

is

a

critical

component

of

the

processing

pipeline,

allowing

accurate

estimate

and

detection

of

DE

genes

However,

the

procedure

for

generating

RNA-seq

data

is

fundamentally

different

from

that

for

microarray

data,

the

normalization

methods

used

in

microarray

data

are

therefore

not

directly

applicable

in

RNA-seq

data..

Current

RNA-seq

analysis

methods

typically

standardize

data

between

samples

by

scaling

the

number

of

reads

in

a

given

library

to

a

common

value

across

all

sequenced

libraries

in

the

experiment.

Several

researchers

have

modeled

the

observed

counts

for

a

gene

with

a

mean

that

includes

a

factor

for

the

total

number

of

reads

Similarly,

for

LONGSAGE-seq

data,

t

Hoen

et

al.

used

the

square

root

of

scaled

counts,

Vencio

et

al.

proposed

a

beta-binomial

model

to

normalization.

Mortazavi

et

al.

adjusted

their

counts

to

reads

per

kilobase

per

million

mapped

(RPKM).

Cloonan

et

al.

log-transformed

the

gene

length-normalized

count

data

and

applied

standard

microarray

analysis

techniques

(quantile

normalization

and

moderated

t-statistics)..

Here,

we

search

a

better

normalization

procedure

which

focus

on

two

main

questions:

Does

the

normalization

improve

DE

detection

(sensitivity)

in

reducing

the

false

discover

rate.

Does

the

normalization

result

in

low

technical

variability

across

replicates

(specificity)?

The

standard

procedure

is

to

compute

the

proportion

of

each

gene's

reads

relative

to

the

total

number

of

reads,

and

compare

that

across

all

libraries,

either

by

transforming

the

original

data

or

by

introducing

a

constant

into

a

statistical

model.

Robinson

et

al.

proposed

a

scale

normalization

(TMM)

method

which

is

two-side

symmetry

trimmed

log-fold-changes.

Compared

to

the

previous

normalization,

the

method

shows

improved

results

for

inferring

differential

expression

in

simulated

and

real

data.

But

the

TMM

method

can

not

normalize

the

data

reasonable

when

the

data

are

asymmetric,

especially,

when

the

proportion

of

DE

genes

is

large

(Additional

file

Figure

Exclusion

of

most

of

genes

may

lead

to

lost

of

too

much

information

and

the

TMM

normalization

scale

is

estimated

by

a

symmetry

trimmed

will

bring

biased

results

when

the

data

are

asymmetric.

We

develop

a

new

method

with

an

iteration

median

of

M-values

(IMM)

to

normalize

the

samples

of

different

sequence

depths.

The

IMM

method

normalizes

the

libraries

without

a

symmetry

trimmed.

The

aim

of

iteration

process

of

IMM

method

is

to

look

for

an

invariant

set

of

non-DE

genes

and

use

the

invariant

set

to

normalize

the

samples..

The

rest

of

this

paper

is

organized

as

follows.

In

Section

'Results

and

discussion',

we

introduce

the

TMM

normalization

method

and

propose

an

iteration

normalization

method

(IMM)

for

detecting

DE

genes.

We

carry

out

extensive

simulation

studies

in

Section

'Simulation

studies'.

In

Section

'Application

to

real

dataset',

we

illustrate

our

method

by

analyzing

a

liver

and

kidney

dataset.

Finally,

some

conclusions

are

drawn.

Normalization

will

be

crucial

in

many

other

applications

of

high

throughput

sequencing

where

the

DNA

or

RNA

populations

being

compared

differ

in

their

composition.

Similar

to

previous

high

throughput

technologies

such

as

microarrays,

normalization

is

an

essential

step

for

inferring

true

differences

in

expression

between

samples.

The

number

of

reads

for

a

gene

is

dependent

not

only

on

the

gene's

expression

level

and

length,

but

also

on

the

population

of

RNA

from

which

it

originates.

We

present

a

straightforward

and

effective

empirical

method

for

normalization

of

RNA-seq

data..

The

IMM

normalization

is

an

effective

and

robust

method

for

estimating

relative

RNA

production

levels

from

RNA-seq

data.

The

IMM

method

estimates

scale

factors

between

samples

that

can

be

incorporated

into

currently

used

statistical

test

methods

for

DE

analysis.

In

our

experience,

the

iterate

will

converge

in

no

more

than

five

steps.

In

essence,

both

microarray

and

TMM

normalization

assume

that

the

majority

of

genes,

common

to

both

samples,

are

not

differentially

expressed.

Our

simulation

studies

indicate

that

the

IMM

method

is

robust

under

the

assumption

that

the

rate

of

DE

genes

is

no

more

than

The

IMM

use

the

median

fold-changes

as

the

normalization

scaling

factor

is

a

robust

method.

As

the

TMM

normalization

method,

the

IMM

also

trim

some

genes

as

DE

genes.

But

the

IMM

method

is

not

just

simply

symmetry

trimmed

up

and

down

side

log-fold-changes.

The

iterated

excluding

the

DE

genes

may

be

more

reasonable

than

the

TMM

method

when

the

log-fold-changes

are

biased

and

the

rate

of

DE

genes

is

large..

From

the

simulation

results,

IMM

normalization

is

more

robust

than

TMM.

In

order

to

investigate

IMM

normalization

in

real

data,

we

use

different

normalization

methods

while

the

same

test

method

to

detect

DE

genes.

And

we

find

that

DE

genes

detected

by

IMM

are

more

consistent

with

gene

expression

level

reported

by

bodymap

data

than

DE

genes

detected

by

IMM.

What's

more,

we

find

DE

genes

identified

only

by

IMM

are

more

likely

related

to

liver

or

kidney

tissue

than

DE

genes

detected

by

TMM.

Therefore,

IMM

normalization

method

is

a

useful

method

in

RNA-seq

data

analysis

for

biologists.

O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O B O B O O O O O O O B O O O O


Analogical insight: toward unifying categorization and analogy Abstract The purpose of this paper is to present two kinds of analogical representational change, both occurring early in the analogy-making process, and then, using these two kinds of change, to present a model unifying one sort of analogy-making and categorization. The proposed unification rests on three key claims: (1) a certain type of rapid representational abstraction is crucial to making the relevant analogies (this is the first kind of representational change; a computer model is presented that demonstrates this kind of abstraction), (2) rapid abstractions are induced by retrieval across large psychological distances, and (3) both categorizations and analogies supply understandings of perceptual input via construing, which is a proposed type of categorization (this is the second kind of representational change). It is construing that finalizes the unification. My goal here has been to describe analogical insight using several key ingredients. One is rapid abstraction during retrieval via the Liberman-Trope mechanism. Another is construing. The rapid abstraction-construing theory of analogy unites categorization and analogy-making by rendering the latter a species of the former. The conclusions to draw from this view of analogy are that spontaneous analogical reminding produces insight and new knowledge, that it does this in part because of a special kind of representational change, namely, rapid abstraction, that this kind of representational change occurs quickly, at the time the analogy is forged, and that construing is what does the final unification.. Rutherford once said: ''In science there is only physics; all the rest is stamp collecting.'' It is ironic, therefore, that we are using Rutherford's analogical insight, his thoughts, and his cognitive representations-his mind-to construct a scientific theory about rapid human analogy making, a part of human psychology, neither of which are physics, nor stamp collecting. We can construe Rutherford, having his analogical insight, as a wayward alpha particle, revealing the nature of the mind. Hopefully, he's not too upset. 20

O B O O B O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O B I O O B O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O


A Multimodel Approach for Complex Systems Modeling based on Classification Algorithms Abstract: In this paper, a new multimodel approach for complex systems modeling based on classification algorithms is presented. It requires firstly the determination of the model-base. For this, the number of models is selected via a neural network and a rival penalized competitive learning (RPCL), and the operating clusters are identified by using the fuzzy K-means algorithm. The obtained results are then exploited for the parametric identification of the models. The second step consists in validating the proposed model-base by using the adequate method of validity computation. Two examples are presented in this paper which show the efficiency of the proposed approach. The multimodel approach has been recently developed in several science and engineering domains, with typical applications in the mechanical and chemical engineering areas, with application to modelling, control and/or fault detection e.g. [1] [2] [3] . It was introduced as an efficient and powerful method to cope with modelling and control difficulties when complex non linear and/or uncertain processes are concerned. The multimodel approach assumes that it is possible to replace a unique non linear representation by a combination of simpler models thus building a so-called model-base. Usually, each model of this base describes the considered process at a specific operating point. The interaction between the different models of the base through normalized activation functions allows the modelling of the global non-linear and complex system. Therefore, the multimodel approach aims at lowering the system complexity by studying its behavior under specific conditions. The multimodel principle is given in figure 1 .. The different models of the base could be of different structures and orders but no model can represent the system in its whole operating domain. The decision unit allows the estimation of the weight of each model and thus the selection of the most relevant models at each time. As for the output unit, controlled by the decision unit, it allows the computation of the multimodel output which is obtained by the contribution of the different models' outputs. In spite of its success in many fields (academic, biomedical, etc), the multimodel approach remains confronted with several difficulties, in particular the determination of the number and parameters of the different models representative of the system and the choice of an adequate method of validities computation used for multimodel output deduction. Last years, many authors [4] [5] [6] [7] have been interested by this approach. The main differences between the proposed studies are the selected method for models identification and the type of models. Linear models are mainly used, e.g. fuzzy Takagi-Sugeno models [8, 9] which are often obtained using linearization methods. However, the multimodel identification is more difficult to work out when the models of the base should be determined from only input/output data. Some results are given in [10] . In the latter case, the number of models must firstly be determined. Then, the data are classified and the models' parameters are estimated. Last, the validities of the different models are computed according to the adequate method. The issue of the selection of the appropriate method of validities' estimation was discussed in [11] . Many authors propose to apply classification algorithms in order to handle a set of dynamical models. For example, neural networks have been used to represent and control complex systems [7, [12] [13] [14] [15] . In another hand, thanks to their ability to classify data and their simplicity, K-means algorithms have proved to be efficient for data clustering e.g. [16, 17] . In short, whereas many architectures using multiple models and neural networks have been proposed, there has not been much work on clustering techniques, based on neural networks and K-means algorithms [18] , applied to traditional multimodel representation using only input/output data. The most tedious issues are related to the model-base size and the clustering procedure which aims to the determination of the operating domains. This paper thus proposes a multimodel approach for complex processes modelling based on classification algorithms. The efficiency of the proposed study is illustrated by two examples for which simulations are proposed. The issue of the determination of the number of models in the base is solved through the application of the Rival Penalized Competitive Learning (RPCL), which is an extension of the work in [19] . In the following sections, the different steps of model-base building are detailed knowing that only input/output data are available. In this paper, a new approach for complex systems modelling is proposed and an experimental validation is presented. This approach is applicable when dealing with complex, strongly nonlinear and uncertain systems. It allows the determination of the base of models, which are representative of the system in different operating domains, by using two classification algorithms and two methods of structural and parametric identification. The issue of selection of the models' number is solved by using a neural network and a Rival Penalized Competitive Learning (RPCL). Once this number determined, the data collected on the system are then clustered by using the fuzzy K-means algorithm. The classification results are exploited for the identification of models. Finally, a validation procedure is worked out in order to demonstrate the ability of the proposed modelling structure to reproduce the system response in different operating domains. The proposed approach has been implemented and applied for a complex mechanical system and for a bioreactor. The obtained results seem to be interesting and prove the efficiency of the proposed modelling strategy.

O B O O B I O O O B O O O O O O O B O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O