A generalization of the adaptive rejection sampling algorithm Abstract Rejection sampling is a well-known method to generate random samples from arbitrary target probability distributions. It demands the design of a suitable proposal probability density function (pdf) from which candidate samples can be drawn. These samples are either accepted or rejected depending on a test involving the ratio of the target and proposal densities. The adaptive rejection sampling method is an efficient algorithm to sample from a log-concave target density, that attains high acceptance rates by improving the proposal density whenever a sample is rejected. In this paper we introduce a generalized adaptive rejection sampling procedure that can be applied with a broad class of target probability distributions, possibly nonlog-concave and exhibiting multiple modes. The proposed technique yields a sequence of proposal densities that converge toward the target pdf, thus achieving very high acceptance rates. We provide a simple numerical example to illustrate the basic use of the proposed technique, together with a more elaborate positioning application using real data. Rejection sampling (see, e.g., Robert and Casella 2004, Chap. 2) is a standard Monte Carlo technique for universal sampling. It can be used to generate samples from a target probability density function (pdf) by drawing from a proposal density. The sample is either accepted or rejected by an adequate test of the ratio of the two pdf's and it can be proved that accepted samples are actually distributed according to the target density (Robert and Casella 2004) . Rejection sampling can be applied as a tool by itself, in problems where the goal is to approximate integrals with respect to (w.r.t.) the pdf of interest, but more often it is a useful building block for more sophisticated Monte Carlo procedures (Gilks and Wild 1992; Gilks et al. 1994; Künsch 2005 ). An important limitation of rejection sampling methods, however, is the need to analytically establish a bound for the ratio of the target and proposal densities. There is a lack of general methods for the computation of exact bounds and so they have to be found for each specific problem at hand.. One exception to this rule is the so-called adaptive rejection sampling (ARS) method (Gilks and Wild 1992; Gilks 1992; Robert and Casella 2004) which, given a target density, yields both a suitable proposal pdf (easy to draw from) and an upper bound for the ratio of the target density over this proposal. The class of adaptive rejection sampling methods is particularly interesting because they ensure high acceptance rates. Indeed the standard ARS algorithm of Gilks and Wild (1992) yields a sequence of proposal functions that actually converge toward the target pdf when the procedure is iterated. As the proposal density becomes closer to the target pdf, the proportion of accepted samples grows. Unfortunately, this algorithm can only be used with log-concave target densities. An extension of the standard ARS has been proposed in Hörmann (1995) , where the same method of Gilks and Wild (1992) is applied to T -concave densities, with T being a monotonically increasing transformation, not necessarily the logarithm. The technique cannot be applied to multimodal densities either.. Indeed, when the target pdf presents several modes, an elaborate extension of the original ARS method is needed. The adaptive rejection Metropolis sampling (ARMS) procedure of Gilks et al. (1995) uses a Metropolis-Hastings algorithm to account for multiple modes. However, this approach is not fully satisfactory because the generated samples form a Markov chain. Hence, they are correlated and, for certain distributions, the chain can be easily trapped in a single mode.. The method in Evans and Swartz (1998) extends the technique of Hörmann (1995) to multimodal distributions. It involves the decomposition of the T -transformed density into pieces which are either convex and concave on disjoint intervals and can be handled separately. Unfortunately, this decomposition requires the ability to find all the inflection points of the T -transformed density, which can be something hard to do for practical problems.. More recently, it has been proposed to handle multimodal distributions by decomposing the log-density into a sum of concave and convex functions (Görür and Teh 2009) . Then, every concave/convex element is handled using a method similar to the ARS procedure. A limitation for the applicability of this technique is the need to decompose the logarithm of the target pdf into concave and convex components, since no general procedure to carry out this decomposition is available. Also, the application of this technique to distributions with an infinite support requires that the tails of the log-densities be strictly concave.. In this paper, we introduce a generalization of the ARS method that can be applied to a large class of target pdf's, possibly not log-concave and possibly multimodal. We assume that the log-density log[p(x)] can be expressed as a sum of composed functions, log[p(x)] = − n i (V i • g i )(x) + cst, where theV i 's are convex and the g i 's are either convex or concave. 1 This is not a universal decomposition that can be applied to every density of interest, but the freedom in the choice of theV i 's and the g i 's enables to describe a family of pdf's that includes, e.g., a posteriori distributions of random variables given a set of independent observations. Further remarks on the applicability of this approach are provided in Sect. 3.6.. The method is based on constructing piecewise-linear approximations of the nonlinearities g i underlying the target density. The construction of these approximations requires a sequence of calculations that can be relatively long depending on the target pdf. They are very systematic though, and the resulting piecewise-linear approximation yield an easyto-sample proposal pdf. In a way also similar to the original ARS, the proposals are improved every time a candidate sample is rejected.. A major difference between the proposed method and the previous approaches, including both original methods by Gilks et al. (Gilks and Wild 1992; Gilks 1992 ) and posterior work on multimodal distributions (Evans and Swartz 1998; Görür and Teh 2009) , is that we do not attempt to construct bounds for the log-or T -transformed density directly. Instead, we build approximations of the nonlinearities g i . In the cases in which the latter functions can be identified, this approach is systematic because every non-linearity is handled using the same approximation procedure (as described in Sect. 3.2 and Appendix). One advantage of the proposed technique, when compared to the procedure of Görür and Teh (2009) , is that it can be extended to handle target densities with an infinite support and tails which are possibly not log-concave, as shown in Sect. 4.3.. The new generalized ARS technique is conceived to be used within more elaborate Monte Carlo methods. It enables, for instance, a systematic implementation of the accept/reject particle filter of Künsch (2005) . There is another potential application in the implementation of the Gibbs sampler for systems in which the conditional densities are complicated, as illustrated in the example of Sect. 5.2. The latter involves the Bayesian estimation of the position of an object using real signal-strength data collected using a network of wireless sensors. Besides the ARS-based implementation of the Gibbs sampler, this example is relevant because it shows how the proposed technique fits particularly well with Bayesian inference problems.. The rest of the paper is organized as follows. Background material is presented in Sect. 2. The basic form of the new algorithm is introduced in Sect. 3, with further extensions in Sect. 4. Section 5 is devoted to the examples. We conclude with a brief summary and conclusions in Sect. 6. We have run the Gibbs sampler (using the GARS algorithm to sample from the conditional pdf's) with three different data sets y. In the first one we collected M = 1 observation per sensor, in the second one we recorded M = 3 observations per sensor and, finally, we obtained a data set with M = 10 measurements per sensors. The target was placed at x = [2.5, 2].. The average acceptance rate of the GARS algorithm was ≈ 30% with M = 1, ≈ 37% with M = 3 and ≈ 26% with M = 10. Note that these rates are, indeed, averages, because the target pdf's are different at each step of the Markov chain (e.g., if. The acceptance rates can be further improved by including additional support points in the initial set S 0 .. Figures 7(a) and (d) show the shape of the true target density p(x 1 , x 2 |y) with M = 1 and M = 3, respectively. Figures 7(b) and (e) depict the corresponding histograms using N = 30,000 samples. We observe that they approximate closely the shape of target pdf's. Finally, Figures 7(c) and (f) illustrate the normalized histograms corresponding to the number of proposed candidates which are needed to accept one sample. Note that we obtain an empirical approximation of the posterior distribution that is very accurate although in terms of the localization accuracy the performance is relatively poor, as there is a bias between the mode of p(x|y) and the actual target position. We have carried out an experiment with a network consisting of four nodes. Three of them are placed at fixed positions and play the role of sensors that measure the strength of the radio signals transmitted by the target. The other node plays the role of the target to be localized. All nodes are bluetooth devices (Conceptronic CBT200U2A) with a nominal maximum range of 200 m.. The deployment of the network is sketched in Fig. 6(a) . We consider a square monitored area of 4 × 4 m 2 and place the sensors at fixed positions h 1 = [h 1,1 = 0.5, h 1,2 = 1], h 2 = [h 2,1 = 3.5, h 2,2 = 1] and h 3 = [h 3,1 = 2, h 3,2 = 3], with all coordinates in meters. The target is located at x = [x 1 = 2.5,. The measurement provided by the ith sensor is denoted as a random variable Y i . To describe the relationship between the observed radio signal strength, Y i = y i , and the target position, denoted by the random vector X = [X 1 , X 2 ], we use the free space propagation model (Rappaport 2001 ). where the norm. is the distance between the ith sensor and the target, γ is a parameter that depends on the physical environment (for open space, γ ≈ 2) and the constant l is the mean power received by each sensor when the target is located at a ref- have been fitted by least squares regression using 200 measurements with the target placed at known distances from each sensor. As a result, we have obtainedl = −27.08,γ = 1.53 andσ = 4.41. Figure 6 (b) depicts the measurements at several distances and the fitted curvel − 10γ log[ Assume we collect M independent measurements from each sensor using the experimental setup just described. . In order to perform inference on the position of the target, we aim at drawing from the posterior pdf. where p(x) is the prior pdf of the target position X. We as-. We apply the Gibbs sampler to draw N particles, denoted. The algorithm can be summarized as follows:. 1. Set j = 1, and draw x (1) 2 from the prior pdf p(x 2 ). 2. Draw a sample x (j ) 1 from the conditional pdf p(x 1 |y,. 2 ), and set The Markov chain generated by the Gibbs sampler converges to a stationary distribution with pdf p(x 1 , x 2 |y). In order to use Gibbs sampling however, we have to be able to draw from the conditional densities p(x 1 |y, x. 2 ) and p(x 2 |y, x (j ) 1 ). In general, these two conditional pdf's can be non-log-concave and can have several modes.. Next, we show how both p(x 1 |y, x. 2 ) and p(x 2 |y, x (j ) 1 ) can be written using the potential-function notation in this paper, in order to sample from them using the proposed GARS method. Specifically, if we let. and the functionsV i (g i (x k )) have the form. with k = 1, 2, and the integers q ∈ {1, 2, 3} and m ∈ {1, . . . , M} are such that i = (q − 1)M + m (in order to enumerate the elements in vector Y). Finallȳ. Therefore, the vector g i consists of 3M nonlinearities. for q = 1, 2, 3 (sensors), m = 1, . . . , M (measurements), k = 1, 2 and j = 1, . . . , N, plus one extra linear function. Note that all the marginal potentials are purely quadratic functions, i.e.,V i (ϑ) = 1 σ 2 ϑ 2 for i = 1, . . . , 3M, and. The potential functions V (x k ; g), k = 1, 2, are not convex in general. Their shape depends on the data set Y = y and the fixed coordinates x (j ). 2 . Therefore, the ARS method cannot be applied to implement steps (2) and (3) of the Gibbs sampler. However, all the marginal potentials are convex and the support of the nonlinearities g i (x k ), i = 1, . . . , 3M + 1, can be partitioned as described in Sect. 4.2. As a consequence, we can use the proposed GARS technique to implement the Gibbs sampler. On the contrary, the form of (35), (36) and (37) makes the calculation of the inflection points (with respect x k ) intractable and, therefore, the methods of Evans and Swartz (1998) and Görür and Teh (2009) are not applicable in this example.

O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Acyclic coloring with few division vertices Abstract. An acyclic k-coloring of a graph G is a mapping φ from the set of vertices of G to a set of k distinct colors such that no two adjacent vertices receive the same color and φ does not contain any bichromatic cycle. In this paper we prove that every triangulated plane graph with n vertices has a 1-subdivision that is acyclically 3-colorable (respectively, 4-colorable), where the number of division vertices is at most 2n − 5 (respectively, 1.5n − 3.5). Our results imply O(n log 16 n) and O(n log 24 n) upper bounds on volume of 3D polyline drawings of planar graphs, where each edge has at most one bend and the total number of bends are 2n − 5 and 1.5n − 3.5, respectively. On the other hand, we prove an 1.28n (respectively, 0.3n) lower bound on the number of division vertices for acyclic 3-colorings (respectively, 4-colorings) of triangulated planar graphs. Furthermore, we establish the NP-completeness of deciding acyclic 4-colorability for graphs with the maximum degree 5 and for planar graphs with the maximum degree 7. A k-coloring of a graph G is a mapping φ from the set of vertices of G to a set of k distinct colors such that no two adjacent vertices receive the same color. We call φ an acyclic k-coloring if it does not contain any bichromatic cycle. The acyclic chromatic number of a graph G is the minimum number of colors required in any acyclic coloring of G.. Grünbaum [1] first introduced the concept of acyclic coloring in 1973 and proved that every planar graph admits an acyclic 9-coloring. Then Mitchem [2] , Albertson and Berman [3] , Kostochka [4] and finally Borodin [5] improved this upper bound to 8, 7, 6 and 5, respectively. Since there exist planar graphs requiring 5 colors in any acyclic coloring [1] , much research effort has been devoted to characterize planar graphs that are acyclically 3 or 4-colorable [6] . Both the problems of deciding acyclic 3 and 4-colorability are NP-hard for planar graphs with maximum degree 4 and 8, respectively [7, 8] .. Grünbaum also considered acyclic colorings of non-planar graphs. He proved that any graph of maximum degree 3 admits an acyclic 4-coloring. Alon et al. [9] gave an O(∆ 4/3 ) upper bound and an Ω(∆ 4/3 /(log ∆) 1/3 ) lower bound on acyclic chromatic number for the graphs with maximum degree ∆. The currently best known upper bounds on acyclic chromatic numbers for the graphs with maximum degree 3, 4, 5 and 6 are 4, 5, 7 and 11, respectively [10] [11] [12] [13] . Little is known about the time complexity of deciding acyclic colorability for bounded degree graphs. Acyclic 3-colorability (respectively, 4-colorability) is NP-complete for bipartite planar graphs with maximum degree 4 (respectively, maximum degree 8) [8] . Recently, Mondal et al. [14] proved that acyclic 4-colorability is NP-complete for the graphs with maximum degree 7.. A k-subdivision of a graph G is a graph G obtained by replacing every edge of G with a path that has at most k internal vertices. We call these internal vertices the division vertices of G. Wood [15] observed that every graph has a 2-subdivision that is acyclically 3-colorable. Angelini and Frati [7] proved that every triangulated planar graph with n vertices has a 1-subdivision with 3n − 6 division vertices that is acyclically 3-colorable. This upper bound on the number of division vertices reduces to 2n − 6 in the case of acyclic 4-coloring [14] .. Acyclic colorings of graphs and their subdivisions find applications in diverse areas [16] [17] [18] . For example, an acyclic coloring of a planar graph has been used to obtain upper bounds on the volume of a 3-dimensional straight-line grid drawing of a planar graph [16] . Consequently, an acyclic coloring of a planar graph subdivision can give upper bounds on the volume of a 3-dimensional polyline grid drawing, where the number of division vertices gives an upper bound on the number of bends sufficient to achieve that volume. The acyclic chromatic number of a graph helps to obtain an upper bound on the size of "feedback vertex set" of a graph, which has wide applications in operating system, database system, genome assembly, and VLSI chip design [17] . Acyclic colorings are also used in efficient computation of Hessian matrix [18] .. In this paper we examine acyclic colorings of 1-subdivisions of planar graphs. We also show some improvement over the previous NP-completeness results in terms of maximum degree. Our results are as follows.. -In Section 3 we prove that every triangulated plane graph with n vertices has a 1-subdivision with at most 2n − 5 (respectively, 1.5n − 3.5) division vertices that is acyclically 3-colorable (respectively, 4-colorable), which significantly improves the previously best known upper bounds 3n − 6 and 2n − 6 on the number of division vertices presented in [7, 14] .. -In Section 4 we establish a 1.28n (respectively, 0.3n) lower bound on the number of division vertices for acyclic 3 and 4-colorings of triangulated planar graphs, respectively.. -In Section 5 we show that deciding acyclic 4-colorability is NP-complete for graphs with maximum degree 5 and for planar graphs with maximum degree 7. Our results improve the previously known NP-completeness results on acyclic 4-colorability for graphs with maximum degree 7 [14] and for planar graphs with maximum degree 8 [8] . The question "What is the minimum positive constant c such that every triangulated planar graph with n vertices has an acyclic k-coloring, k ∈ {3, 4}, with at most cn division vertices?" was posed in the 22nd International Workshop on Combinatorial Algorithms (IWOCA 2011) [23] . Although we proved that 1.28 ≤ c ≤ 2 and 0.3 ≤ c ≤ 1.5 for k = 3 and k = 4, respectively, there is a gap between the upper bound and the lower bound leaving a scope for improvement.. Proof of Remark 1.. Proof. Suppose for a contradiction that there is a bichromatic cycle C in G . C cannot be a cycle in G because the coloring in G is bichromatic. Then C must contain the path u, w 1 , . . . , w j , v. Since the vertices on the path u, w 1 , . . . , w j , v receives three different colors in φ, C cannot be a bichromatic cycle. Therefore, φ is an acyclic coloring of G .

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O B I O O O O B I O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O


The asymptotic behavior of some indices of iterated line graphs of regular graphs In this paper, we consider the asymptotic behavior of the number of spanning trees and the Kirchhoff index of iterated line graphs and iterated para-line graphs (or clique-inserted graphs) of a regular graph G. We show that the asymptotic behavior of these indices (except the Kirchhoff index of the iterated para-line graphs) is independent of the structure of the regular graph G. A line graph is a classical unary operation of graphs with finite number and infinite number of vertices. Its basic properties can be found in any text book on graph theory (see for example [1, 3, 6] ). The concept of the para-line graph (or clique-inserted graph) of a graph is relatively new. The definition of a para-line graph was first introduced in [12] . The author obtained the spectrum of the para-line graph of a regular graph G with infinite number of vertices in terms of the spectrum of G. In [19] , the para-line graph is called the clique-inserted graph. The authors showed that the limits of the spectrum of iterated para-line graphs of a regular graph G are a fractal, which is independent of the structure of G.. It is well known that the operation of a line graph can produce many new types of graphs. For example, taking the line graphs of honeycomb lattices with free, cylindrical, and toroidal boundaries, we get Kagomé lattices with free, cylindrical, and toroidal boundaries (see [16] ). The line graph of a tetrahedron (respectively, cube and dodecahedron) graph is an octahedron (respectively, cuboctahedron and icosidodecahedron) graph. Clearly, we can go further, but iterated line graphs are not polyhedron graphs (3-connected planar graphs). On the other hand, the para-line graphs of honeycomb lattices with free, cylindrical, and toroidal boundaries are the 3.12.12 lattices with free, cylindrical, and toroidal boundaries (see [16] ). This iterated procedure can be carried out repeatedly to produce a series of cubic lattices. For cubic polyhedron graphs such as the tetrahedron, cube, and dodecahedron, the iterated procedure produces infinite families of truncated polyhedron graphs.. In this paper, we discuss the growth rate of the number of spanning trees and the Kirchhoff index of iterated line graphs and iterated para-line graphs of a regular graph G. The study of enumeration of spanning trees has a long history in both mathematics and physics. Motivated by problems in the real world, the classical work of Kirchhoff [9] and Cayley [4] considered the problem of enumeration of spanning trees of graphs. Now, this has become basic material in many text books on graph theory. The concept of the Kirchhoff index appeared relatively late. It had been introduced in chemistry by Klein and Randić [10] . For further work on the Kirchhoff index, see, for example, [5, 8, 15, 17, 18, [20] [21] [22] [23] .. In Section 2, we determine the growth rate of the number of spanning trees of iterated line graphs and iterated para-line graphs of an r-regular graph G. We prove that their growth rates are independent of the structure of G and only dependent on r and the number of vertices of G. In Section 3, we determine the growth rate of the Kirchhoff index of iterated line graphs and iterated para-line graphs of a regular graph. We show that the growth rate of the Kirchhoff index of iterated line graphs of an r-regular graph G is independent of the structure of G and only dependent on r and the number of vertices of G; however, the growth rate of the Kirchhoff index of iterated para-line graphs of an r-regular graph G is indeed dependent on the structure of G.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O


Varieties of Helmholtz Machine The Helmholtz machine is a new unsupervised learning architecture that uses topdown and bottom up connections to build probability density models of input and inverses to those models. The wake-sleep learning algorithm for Helmholtz machines involves just the purely local delta rule. This paper suggests a number of different varieties of Helmholtz machines, each with its own strengths and weaknesses, and draws conclusions from the hypothesis that cortical information processing can be viewed in their terms. This special issue focuses on four important questions about neural computation: 1) How is the world represented in the firing of neurons? 2) What are the functional roles of bottom-up and top-down connections between cortical areas? 3) Does the brain use internal models of the external world? 4) What are the basic synaptic plasticity rules? We believe that answers to all four of these questions are indicated by the new theory of unsupervised learning that we present here.. Supervised learning algorithms for neural networks usually have the clear and easily justified goal of minimizing some loss function (eg the expected squared error or misclassification rate) on test data that is drawn from the same distribution as the training data. Complications may arise because the test data is not available during training, but at least the goal is clear. For unsupervised learning algorithms the goals are often less clear and less justifiable. Goals that have been proposed in the literature can be roughly divided into four broad classes:. Revealing structure in the ensemble of input vectors. This can be done by assigning each input vector to a cluster or to a location in a low-dimensional map. It can also be done by redescribing the input vectors in terms of multiple hidden causes that are not mutually exclusive, or by directly extracting informationally rich relationships between different parts of the input (Becker & Hinton, 1992) .. Preprocessing the data to facilitate a subsequent stage of supervised or reinforcement learning. The preprocessing may reduce the dimensionality of the data (eg principal components analysis, PCA, or self-organising feature maps, Kohonen, 1982) . It may also produce a representation in terms of approximately independent features, as in PCA or factorial codes (Barlow, 1989) which typically reduces interference and increases the convergence speed of subsequent supervised learning procedures. It may even allow the subsequent supervised stage to be linear (as in radial basis functions).. Assigning a probability density to input vectors so as to maximize the likelihood of the observed data or the expected likelihood of more data from the same distribution. Density estimation allows novel input vectors to be detected by their low probability density. It also allows input vectors to be assigned to classes probabilistically if a separate density model is created for each class.. Compressing the data, with or without loss, to facilitate storage or communication.. Obviously, these goals are not independent. For example, clustering the data using a mixture of Gaussians model also assigns a probability density to each input vector. Using a hierarchical coding scheme to compress input vectors can be a good way of discovering true hidden causes because, in the large data limit, the true way in which the data was generated is almost certainly the most efficient way to code it. Discovering the true hidden causes of the input vectors is an excellent way to facilitate subsequent supervised learning because desired responses are almost always more simply related to the underlying causes of the sensory data than to the raw sensory data itself. Schemes for losslessly compressing input vectors can be used as probability density estimators because the number of bits required to code an input vector losslessly can always be viewed as an upper bound on its negative log probability.. Helmholtz machines, the new model of unsupervised learning which we present here, were originally motivated by the idea that hierarchical compression schemes would reveal the true hidden causes of the sensory data and that this would greatly facilitate subsequent supervised learning. An obvious practical advantage of this two stage approach is that it is typically easier to get the unlabelled data needed for unsupervised learning than the labelled data required for supervised learning, so the first stage can extract useful information from data that is useless for pure supervised learning. It is interesting, but far less obvious, that under very reasonable assumptions, the two stage approach can be superior even if all of the data is labelled. The minimum description length framework (MDL, Rissanen, 1989) can be used to clarify this point. For a neural network to be of any value at all, the information required to specify the parameters of the neural network must be less than the information saved when communicating the data using the network. When MDL is applied to supervised learning, it is the labels that need to be communicated, so the information that each training case provides about the parameters is at most the the number of predictable bits in the label. In unsupervised learning it is the input vectors that must be communicated so the useful information is at most the redundancy in the input vector which is typically much greater. Unsupervised learning therefore extracts far more information per training case. Of course, this information may be irrelevant to the labels. In the worst case, a complicated generative process involving hidden states could be used to model the input vectors, but the label associated with each input vector could have nothing to do with the hidden states of the generative model. Luckily, the world is not like this. It is almost always true that the desired responses to raw sensory data have a simpler relationship to the way the data was generated than to the raw data itself.. The next section develops the theory behind the two existing varieties of Helmholtz machine; section 3 discusses alternative version of the machine that we have explored, changing the internal architecture, the unit activation functions, and the learning rules; section 4 draws together the answers that are suggested by these Helmholtz machines to the four questions about the cortex. We have described the original version of the Helmholtz machine and a number of closely related machines that have been designed with different tasks or mechanisms in mind, eg the supervised machines, the Helmholtz machine through time and the lateral HM. What relevance do any of these machines have to cortical processing and the four questions underlying this special issue?. In our models, the computational goal for unsupervised learning is probability density estimation. Along with Kawato, Hayakama & Inui (1993) , we claim that the cortex builds a complex and hierarchical generative model 6 of its inputs, where the layers in the hierarchy roughly correspond to areas in the sense of Felleman & Van Essen (1991) . The neurons in these areas are the hidden units, and they represent structure in the generative model. For instance, in a generative model of images of digits, hierarchical hidden structure might 6 This model is sometimes called a forward model, but since we see it as being top-down, we have avoided use of this terminology. consist of such choices as which digit, what font, what curlicues, etc. The process of setting the weights in the connections between the neurons (and possibly determining the connections themselves, Friedlander & Martin, 1989; Antonini & Stryker, 1993) to improve the generative model gives the hidden units their functions. Although Barlow has not stressed so greatly the issue of learning probabilistic models of the inputs, the final roles accorded to the hidden units turn out to be closely analogous to his suggestions that neurons should respond to few patterns (be sparse), that neurons with common targets should generally fire in a somewhat independent manner (be factorial or decorrelated), and that they should respond to statistically surprising conjunctions of their inputs. However, all these desiderata are secondary to the main goal of density estimation.. As far as the coding scheme adopted by cortex is concerned, the HM is somewhat agnostic. Much of the work on unsupervised learning as activity dependent development has focused on bottom-up learning, and is driven by objective functions such as the sparseness or mutual independence of activities. Mutual independence is good, for instance, since later stages can combine information without having to build very complicated probability models. However, these metrics are somewhat unsatisfying since they are not couched in terms of a whole goal for development. For instance, there are probably many sources of independent noise afflicting the inputs, but it would be unfortunate were cortex to arrange for mutual independence by filtering out the signal in favour of the noise. By contrast, the HM has optimal density estimation as its underlying motivation -and doing this correctly requires inducing the statistical structure of the inputs. There will be cases in which this will entail that the hidden activities it forms will be sparse, but sparseness itself plays no particular role. Equally, the HM can work with local softmax groups within a layer, which is a popular reading of the effect of short-range cortical connectivity, provided that they are combined in a factorial manner. The lateral HM points towards learning methods for such intra-cortical connections, but needs further exploration.. With Carpenter & Grossberg, (1987) , Kawato et al (1993) and Mumford (1994) , the HM is clearly not so agnostic about the existence of internal models of the world and the role of top-down and bottom-up cortical connections in learning those models and their inverses. In particular, the wake-sleep algorithm can be used to make precise hypotheses about the nature of the learning that occurs in both sorts of connections.. Model building is the force in the HM. There is something slightly unsatisfying about this which emerged in some of the simulations for Hinton et al (1995) . We tried taking images of all ten digits and training one large and deep density model on the whole collection. We had expected that the most significant structure in the input would be the identity of a digit, and that units towards the top of the hierarchy would tend to respond to just one digit, or maybe even at a finer granularity than that -just one style of one digit. Structure in the world, in terms of the way we interact with it, should be matched by statistical structure that the HM or other density estimation devices can extract. However, the HM did not separate out the digits in this manner -top level units were not digit specific. Worse, it turned out not even to be fruitful to attempt to recognise digits from the activities of the hidden units -the HM is apparently capable of extracting the 'wrong' sort of structure. We are presently investigating why.. We have used two main rules for synaptic plasticity -the delta rule (Konorski, 1948; Widrow & Stearns, 1985; Rescorla & Wagner, 1972) in the wake-sleep algorithm and a correlational or Hebbian rule (Hebb, 1949) for the Boltzmann machine learning in the lateral HM. However, our main goal in designing learning rules was to make all the information required for learning be local, a minimal requirement for biological plausibility. Although most of the evidence on neural plasticity points to the implementation of Hebb rules, it is well known that delta-like rules can be implemented by Hebb-like mechanisms (Artola, Bröcher & Singer, 1990; Hancock, Smith and Phillips, 1991; Montague & Sejnowski, 1994; Montague et al, 1994) .. Unsupervised learning of representations is only part of the overall computational economy of the brain -reinforcement and associative learning based on these representations must also play major roles. The HM offers a crisp hypothesis for how density estimation in complex hierarchical models can be implemented and suggests a role for puzzling anatomical motifs such as reciprocal connectivity. How closely the HM maps onto the microcircuitry of cortex is under investigation.

O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On Ostrowski type inequalities for Stieltjes integrals with absolutely continuous integrands and integrators of bounded variation Abstract. Some Ostrowski type inequalities are given for the Stieltjes integral where the integrand is absolutely continuous while the integrator is of bounded variation. The case when |f | is convex is explored. Applications for the midpoint rule and a generalised trapezoid type rule are also presented. The following result is known in the literature as Ostrowski's inequality: Let f : [a, b] → R be a differentiable mapping on (a, b) with the property that |f (t)| ≤ M for all t ∈ (a, b) . Then. for all x ∈ (a, b) . The constant 1 4 is best possible in the sense that it cannot be replaced by a smaller constant.. The above result has been naturally extended for absolutely continuous functions and Lebesgue p−norms of the derivative f in [11] - [13] and can be stated as: They can also be obtained, in a slightly different form, as particular cases of some results established by A.M. Fink in [14] for n−time differentiable functions.. For other Ostrowski type inequalities concerning Lipschitzian and r − H−Hölder type functions, see [8] and [10] .. The cases of bounded variation functions and monotonic functions were considered in [4] and [7] while the case of convex functions was studied in [3] .. In an effort to obtain an Ostrowski type inequality for the Stieltjes integral, which obviously contains the weighted integrals case, S.S. Dragomir established in [5] the following result:. where r ∈ (0, 1] and H > 0 are given. Then, for any . For other results concerning inequalities for Stieltjes integrals, see [1] , [15] and [16] .. The aim of the present paper is to continue the study of Ostrowski type inequalities for Stieltjes integrals b a f (t) du (t) where the function f, the integrand, is assumed to be absolutely continuous while the integrator u, is of bounded variation. Applications to the midpoint rule and for a generalised trapezoid rule are also pointed out.

O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A formal specification framework for object-oriented distributed systems AbstractÐIn this paper, we present the Concurrent Object-Oriented Petri Nets (CO-OPN/2) formalism devised to support the specification of large distributed systems. Our approach is based on two underlying formalisms: order-sorted algebra and algebraic Petri nets. With respect to the lack of structuring capabilities of Petri nets, CO-OPN/2 has adopted the object-oriented paradigm. In this hybrid approach (model-and property-oriented), classes of objects are described by means of algebraic Petri nets, while data structures are expressed by order-sorted algebraic specifications. An original feature is the sophisticated synchronization mechanism. This mechanism allows to involve many partners in a synchronization and to describe the synchronization policy. A typical example of distributed systems, namely the Transit Node, is used throughout this paper to introduce our formalism and the concrete specification language associated with it. By successive refinements of the components of the example, we present, informally, most of the notions of CO-OPN/2. We also give some insights about the coordination layer, Context and Objects Interface Language (COIL), which is built on top of CO-OPN/2. This coordination layer is used for the description of the concrete distributed architecture of the system. Together, CO-OPN/2 and COIL provide a complete formal framework for the specification of distributed systems. F OR important applications, distributed processing provides a general, flexible, and evolutionary approach. Distributed processing offers many advantages: availability and reliability through replication; performance through parallelism; sharing and interoperability through interconnection; flexibility and incremental expansion through modularity.. However, distributed systems introduce several new considerations that must be necessarily taken into account. Interactions between independent components give rise to new issues such as nondeterminism, contention, and synchronization.. In a formal and rigorous specification process, it is necessary to have a sound mathematically-based formalism which allows to express all of the characteristics of the systems. When we are faced with large problems, it is also necessary to have at one's disposal some structuring facilities. Moreover, the ever-increasing complexity of software systems imposes a progressive adaptation based on abstraction, refinement, and enrichment. Thus, specifying a system in a formal and incremental way offers several benefits. Indeed, the initial perception of the system to be built may be very vague. As the analysis and the simultaneous validation progress, the definition of the architecture, of the algorithms, and of the associated data structures gradually improve; finally, the implementation can take form.. In this paper, we present a formal framework for the development of distributed systems. The approach we propose has adopted the object-oriented paradigm as a structuring principle. On the one hand, we have devised a general formalism which can express both abstract and concrete aspects of systems, with emphasis on the description of concurrency and abstract data types. This approach, called Concurrent Object-Oriented Petri Nets (CO-OPN/2) extends its object-based predecessor COOPN [9] . On the other hand, a coordination layer, called Context and Objects Interface Language (COIL), has been developed on top of this formalism in order to be able to deal with distributed architectures. The objective of this paper is to introduce our approach in an intuitive way. Thus, the presentation is organized around expressiveness aspects and methodological considerations rather than theoretical matters which are described in [6] and [7] , with respect to CO-OPN/2, and in [13] for COIL.. For the presentation of our approach, we use a typical example of distributed systems, namely the Transit Node (TNode), and progressively build a distributed communication system composed of TNodes. The presentation is divided into several steps. The first one deals with the abstract data types used in the example and constructs a basic version of the communication system in which all the TNodes can transmit messages to each other. This first simple version allows us to introduce most of the fundamental concepts of CO-OPN/2. The second step enriches the basic system regarding the data input/ output ports. In the third step, the routes are added; the construction of this new system is based on inheritance.. A more realistic system of a heterogeneous distributed system is then introduced. This system takes into account the timing information, the error detection algorithms, and the heterogeneous distributed systems gateway. A last evolution of the communication system introduces the coordination layer COIL to support the design of distributed architectures.. Finally, we discuss some semantic aspects of CO-OPN/2, and present the state of the art of various object-oriented formal methods based on a work that we did with other colleagues on this subject [16] . In this paper, we have presented a formal approach called CO-OPN/2 which can be used for the development of large distributed systems. This approach is flexible enough to be adapted to the modeling of many different kinds of applications (including the development of distributed algorithms [12] , the development of parallel algorithms [8] ) as well as to the semantics of concurrent language (by modeling the actor languages [10] ). The transit node has been used to describe completely and informally particular aspects of the CO-OPN/2 language.. CO-OPN/2 supports modularity and the expression of various kinds of concurrencies. In this paper, we have shown how CO-OPN/2 can handle concurrency issues and classification requirements. The following major points of our approach have been presented: We express intraobject and interobject true concurrency; We differentiate inheritance as a syntactic mechanism from subtyping which is a semantic concern; We give a way to express hierarchies of abstraction; We allow modular construction of specifications; we give a formal model of specification refinement.. Nevertheless, important research topics around CO-OPN/2 have not been presented here, such as verification issues that can be found in [3] , validation issues that are presented in [11] and refinement for distributed system development and verification based on temporal logic that are presented in [14] . Complete CO-OPN/2 semantics is presented in [6] and [7] .. With the progressive development of the Transit node, we have shown the possible ways of refining specifications, starting from an abstract description and progressively introducing concrete aspects. The validation of each step of the refinement process was possible through the use of prototyping or proving tools available in the SANDS environment [8] .. Future studies will be conducted along three directions: Tools to support the new features of the CO-OPN/2 language for the already developed environment SANDS will be developed (you can download the tool at: http:/lglwww.epfl.ch). A distribution model of CO-OPN/ 2 objects will be defined for heterogeneous systems with its operational semantics; Experiment will be performed with CO-OPN/2 on different practical case studies. Nicolas Guelfi received the PhD degree in 1994 from the University of Paris XI-Orsay in France in the field of formal specification of concurrent systems. Since 1999, he has been a full professor at the Applied Computer Science Department at the Luxembourg University of Applied Sciences. In 1994 and 1995, he worked as a research and teaching assistant at the University of Paris XII-Creteil and then joined, for three years, the Software Engineering Laboratory at the Lausanne Swiss Federal Institute of Technology in Switzerland where he has participated to teaching, research projects, and PhD thesis supervision. Since 1990, he has been involved in three European ESPRIT BRA projects (DEMON, CALIBAN, and DEVA), two national research projects in Switzerland and one technology transfer project. All these research activities have been done in the field of formal software engineering methods and tools for distributed systems and have contributed to the CO-OPN framework. Currently, he is the head of the Software Engineering Competence Center and director of the Computer Sciences Studies where he is in charge of managing national and international research projects on scientific methodologies applied to the engineering of distributed systems and databases.

O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O


Autonomic

Provisioning

with

Self-Adaptive

Neural

Fuzzy

Control

for

Percentile-Based

Delay

Guarantee

Autonomic

server

provisioning

for

performance

assurance

is

a

critical

issue

in

Internet

services.

It

is

challenging

to

guarantee

that

requests

flowing

through

a

multi-tier

system

will

experience

an

acceptable

distribution

of

delays.

The

difficulty

is

mainly

due

to

highly

dynamic

workloads,

the

complexity

of

underlying

computer

systems,

and

the

lack

of

accurate

performance

models.

We

propose

a

novel

autonomic

server

provisioning

approach

based

on

a

model-independent

self-adaptive

Neural

Fuzzy

Control

(NFC).

Existing

model-independent

fuzzy

controllers

are

designed

manually

on

a

trial-and-error

basis,

and

are

often

ineffective

in

the

face

of

highly

dynamic

workloads.

NFC

is

a

hybrid

of

control-theoretical

and

machine

learning

techniques.

It

is

capable

of

self-constructing

its

structure

and

adapting

its

parameters

through

fast

online

learning.

We

further

enhance

NFC

to

compensate

for

the

effect

of

server

switching

delays.

Extensive

simulations

demonstrate

that,

compared

to

a

rule-based

fuzzy

controller

and

a

Proportional-Integral

controller,

the

NFC-based

approach

delivers

superior

performance

assurance

in

the

face

of

highly

dynamic

workloads.

It

is

robust

to

variation

in

workload

intensity,

characteristics,

delay

target,

and

server

switching

delays.

We

demonstrate

the

feasibility

and

performance

of

the

NFC-based

approach

with

a

testbed

implementation

in

virtualized

blade

servers

hosting

a

multi-tier

online

auction

benchmark.

Today,

popular

Internet

services

employ

a

complex

multi-tier

architecture,

with

each

tier

provisioning

a

certain

functionality

to

its

preceding

tier

and

making

use

of

the

functionality

provided

by

its

successor

to

carry

out

its

part

of

the

overall

request

processing

[Urgaonkar

et

al.

Autonomic

resource

management

for

Internet

services

aims

to

reduce

the

degree

of

human

involvement

in

the

management

of

complex

computing

systems

[Huebscher

and

McCann

It

is

critical

to

performance

assurance

and

challenging

due

to

rapidly

growing

scale

and

complexity

of

multi-tier

Internet

services.

Dynamic

server

provisioning

with

virtualization

is

essential

to

A

preliminary

version

of

the

article

appeared

in

Lama

and

Zhou

This

research

was

supported

in

part

by

NSF

CAREER

award

and

research

grant

Authors'

address:

P.

Lama

and

X.

Zhou

(corresponding

author),

University

of

Colorado,

Colorado

Springs,

CO;

email:

{plama,

xzhou}@uccs.edu.

Permission

to

make

digital

or

hard

copies

of

part

or

all

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

show

this

notice

on

the

first

page

or

initial

screen

of

a

display

along

with

the

full

citation.

Copyrights

for

components

of

this

work

owned

by

others

than

ACM

must

be

honored.

Abstracting

with

credit

is

permitted.

To

copy

otherwise,

to

republish,

to

post

on

servers,

to

redistribute

to

lists,

or

to

use

any

component

of

this

work

in

other

works

requires

prior

specific

permission

and/or

a

fee.

Permissions

may

be

requested

from

Publications

Dept.,

ACM,

Inc.,

Penn

Plaza,

Suite

New

York,

NY

USA,

fax

or

permissions@acm.org.

c

ACM

P.

Lama

and

X.

Zhou

the

performance

assurance

of

multi-tier

Internet

services

and

resource

utilization

efficiency

of

the

underlying

computer

systems.

Recent

research

efforts

relied

on

queuing-theoretic

approaches

[Padala

et

al.

Urgaonkar

et

al.

Villela

et

al.

and

control-theoretic

approaches

[Kamra

et

al.

based

on

explicit

system

performance

models

for

dynamic

server

allocation

in

multi-tier

systems.

However,

it

is

complex

and

time

consuming

to

accurately

estimate

system

performance

model

parameters

such

as

service

time,

workload

distribution,

etc.

Furthermore,

system

parameter

variation

of

virtual

servers,

nonstationary

workload,

and

inherent

nonlinearity

of

performance

versus

resource

allocation

introduce

additional

challenges

to

achieve

an

accurate

system

performance

model

[Litoiu

End-to-end

system

delay

is

the

major

performance

metric

of

multi-tier

Internet

applications.

It

is

the

response

time

of

a

request

that

flows

through

a

multi-tier

computer

system

[Urgaonkar

et

al.

Figure

depicts

a

typical

three-tier

Internet

service

architecture.

For

load

sharing,

a

tier

is

often

replicated

and

clustered

based

on

server

virtualization

techniques.

The

queueing-model-based

approaches

and

feedbackcontrol-based

approaches

can

guarantee

the

average

delay

of

requests.

But

they

have

no

control

on

an

important

performance

metric,

percentile-based

end-to-end

delay

of

requests.

A

percentile-based

performance

metric

such

as

the

th

-percentile

end-to-end

delay

is

easy

to

reason

about

and

to

capture

individual

users'

perception

of

Internet

service

performance

[Mi

et

al.

Welsh

and

Culler

It

is

very

challenging

to

assure

a

percentile-based

delay

guarantee

of

requests

of

a

multi-tier

Internet

service.

Compared

with

the

average

delay,

a

percentile

delay

introduces

much

stronger

nonlinearity

to

the

system

performance

model.

Queueingtheoretic

techniques

have

achieved

noteworthy

success

in

providing

average

delay

guarantee

on

multi-tier

server

systems.

However,

most

queueing

models

are

mean

oriented.

It

is

costly

to

compute

various

moments

of

the

distribution

of

response

times

from

a

queueing

model.

Control-theoretic

techniques

were

applied

to

inherently

nonlinear

Web

systems

for

performance

guarantees

by

performing

linear

approximation

of

system

dynamics

and

estimation

of

system

parameters.

However,

if

the

deployed

system

configuration

or

workload

range

deviates

significantly

from

those

used

for

system

identification,

the

estimated

system

model

used

for

control

would

become

inaccurate

[Lu

et

al.

Recent

studies

have

seen

highly

dynamic

workloads

of

Internet

services

that

fluctuate

over

multiple

time

scales,

which

can

have

a

significant

impact

on

the

processing

demands

imposed

on

data

center

servers

[Mi

et

al.

[Mi

et

al.

Singh

et

al.

In

this

article,

we

propose

an

autonomic

server

allocation

approach

based

on

a

model-independent

neural

fuzzy

controller

for

percentile-based

end-to-end

delay

guarantee

in

virtualized

multi-tier

server

clusters.

There

are

model-independent

rule-based

fuzzy

controllers

that

utilize

heuristic

knowledge

for

performance

guarantee

on

Internet

servers

[Lama

and

Zhou

Liu

et

al.

Wei

and

Xu

They

use

a

set

of

predefined

rules

and

fuzzy

membership

functions

to

perform

control

actions

in

the

form

of

resource

allocation

adjustment.

These

controllers

have

some

drawbacks.

First,

they

are

designed

manually

on

trial-and-error

basis,

using

heuristic

control

knowledge.

There

is

no

specific

guideline

for

determining

important

design

parameters

such

as

the

input

scaling

factors,

the

rule

base,

and

the

fuzzy

membership

functions.

Second,

those

design

parameters

are

nonadaptive.

They

are

not

effective

in

the

face

of

highly

dynamic

workloads.

To

overcome

the

drawbacks,

we

design

a

novel

self-adaptive

neural

fuzzy

controller

as

a

hybrid

of

control-theoretical

and

machine

learning

techniques..

The

main

advantages

of

the

proposed

server

allocation

approach

based

on

the

neural

fuzzy

controller

are

as

follows..

It

is

robust

to

highly

dynamic

workload

intensity

as

well

as

characteristics

and

change

in

delay

target

due

to

its

self-adaptive

and

self-learning

capabilities.

It

is

model

independent.

The

parameter

variations

of

the

system

performance

and

the

unpredictability

of

dynamic

workloads

do

not

affect

the

validity

and

effectiveness

of

the

proposed

server

allocation

approach.

It

is

capable

of

automatically

constructing

the

control

structure

and

adapting

control

parameters

through

fast

online

learning.

The

controller

executes

resource

allocation

adjustment

and

learns

to

improve

its

performance

simultaneously.

Unlike

other

supervised

machine

learning

techniques

that

learn

from

the

supervised

training

data,

it

does

not

require

offline

training.

Avoiding

offline

training

saves

significant

amount

of

time

and

efforts

required

to

collect

a

large

set

of

representative

training

data

and

to

train

the

system..

Furthermore,

we

address

an

important

server

switching

cost

issue.

Server

switching

by

addition

and

removal

of

a

virtual

server

introduces

nonnegligible

latency

to

a

multitier

service.

It

affects

the

perceived

end-to-end

delay

of

users.

It

takes

time

for

a

newly

added

server

to

adapt

to

the

existing

system.

For

example,

an

addition

of

database

replica

goes

through

a

data

migration

and

system

stabilization

phase

[Chen

et

al.

A

removal

of

a

server

does

not

happen

instantaneously,

since

it

has

to

process

residual

requests

of

an

active

session.

To

compensate

for

the

server

switching

delay,

we

perform

two

enhancements

on

our

neural

fuzzy

controller.

First,

we

incorporate

the

effect

of

server

switching

with

the

online

parameter

learning.

Second,

we

integrate

a

self-tuning

component

that

adjusts

its

output

to

proactively

compensate

for

the

server

switching

effect..

For

performance

evaluation

of

the

proposed

server

provisioning

approach,

we

build

a

simulation

model

and

conduct

extensive

simulations

using

synthetic

heavy-tailed

workloads.

Simulation

results

demonstrate

the

effectiveness

of

the

new

approach

in

achieving

the

end-to-end

delay

guarantee

for

both

stationary

and

highly

dynamic

workloads.

We

apply

dynamic

workloads

with

sudden

step-changes

similar

to

what

was

used

in

Urgaonkar

et

al.

and

with

continuous

changes

similar

to

what

was

used

in

Chen

et

al.

Although

our

approach

is

designed

in

a

way

that

is

applicable

to

any

percentile-based

end-to-end

delay

guarantees,

the

experiments

are

conducted

for

assuring

the

th

-percentile

and

the

median

end-to-end

delay

targets.

Results

show

that

our

approach

is

effective

in

provisioning

percentile-base

delay

guarantee.

We

perform

the

sensitivity

analysis

of

the

neural

fuzzy

controller

for

various

delay

targets

and

compare

its

performance

with

the

rule-based

fuzzy

controller

used

in

Zhou

The

neural

fuzzy

controller

delivers

consistently

better

performance

for

various

delay

targets.

It,

on

average,

outperforms

the

rule-based

fuzzy

control

approach

by

about

and

in

terms

of

relative

delay

deviation

and

temporal

target

violation,

respectively.

We

also

demonstrate

the

impact

of

the

input

scaling

factor

on

the

performance

of

the

rule-based

fuzzy

controller

for

different

delay

targets.

There

does

not

exist

one

single

scaling

factor

that

works

best

for

different

scenarios.

It

demonstrates

the

need

of

self-adaptivity

based

on

the

neural

fuzzy

control.

Then,

we

show

the

robustness

of

the

new

server

provisioning

approach

to

server

switching

delays

and

study

the

impact

of

the

control

interval

on

the

responsiveness

of

control

actions.

Furthermore,

we

demonstrate

that

our

approach

outperforms

a

commonly

used

proportional

integral-control-based

server

provisioning

technique

in

the

face

of

dynamically

varying

workload

intensity

as

well

as

characteristics..

Finally,

we

conduct

a

feasibility

study

with

performance

evaluation

of

the

proposed

server

provisioning

approach

based

on

a

testbed

implementation

of

a

virtualized

server

cluster

hosting

RUBiS

application

[Amza

et

al.

RUBiS

a

multi-tier

online

auction

Web

site

benchmark.

The

testbed

is

built

on

a

cluster

of

HP

ProLiant

blade

server

modules

using

VMware

virtual

machines.

Experimental

results

show

that

the

self-adaptive

neural

fuzzy-control-based

server

provisioning

approach

is

able

to

assure

the

percentile-based

end-to-end

delay

guarantee

in

the

face

of

a

highly

dynamic

workload

with

varying

intensity

and

characteristics..

The

rest

of

this

article

is

organized

as

follows.

Section

reviews

related

work

in

autonomic

resource

provisioning.

Section

presents

the

design

of

self-adaptive

neural

fuzzy

control

for

dynamic

server

provisioning.

Section

describes

the

enhancement

in

the

neural

fuzzy

controller

for

server

switching

delays.

Section

presents

simulation

results

and

performance

evaluation.

Section

presents

the

case

study

based

on

a

testbed

implementation.

Concluding

remarks

and

discussions

about

the

future

work

are

given

in

Section

Performance

assurance

is

critical

and

challenging

to

data

centers

that

host

popular

Internet

services.

In

this

article,

we

have

designed

a

novel

self-adaptive

neural

fuzzycontrol-based

server

provisioning

approach

to

guarantee

the

end-to-end

delay

of

requests

flowing

through

multi-tier

server

clusters.

The

approach

can

effectively

provide

any

percentile-and

the

mean-based

delay

guarantee.

The

major

contributions

lie

in

the

design

and

evaluation

of

a

model-independent

and

self-adaptive

control

system

for

dynamic

server

provisioning.

We

combine

the

strength

of

both

machine

learning

and

control-theoretic

techniques

for

robust

performance

assurance

in

the

face

of

highly

dynamic

workloads.

We

further

enhance

the

neural

fuzzy

controller

to

compensate

for

the

effect

of

server

switching

delays.

We

have

also

studied

the

impact

of

input

scaling

factor

on

controller's

self-adaptivity

and

the

impact

of

the

control

interval

on

the

system

robustness..

Our

approach

is

capable

of

automatically

constructing

the

controller's

structure

and

adapting

control

parameters

through

fast

online

learning.

Simulation

results

have

demonstrated

that

the

neural

fuzzy

controller

is

robust

to

highly

dynamic

workloads

and

changes

in

delay

target.

Compared

to

the

rule-based

fuzzy

controller

and

a

classical

PI

controller,

the

new

approach

has

shown

superior

performance

in

achieving

the

end-to-end

delay

assurance,

particularly

under

highly

dynamic

workloads.

Importantly,

we

have

demonstrated

the

feasibility

and

excellent

performance

of

the

new

approach

in

a

testbed

implementation

of

virtualized

server

cluster.

The

neural

fuzzy

controller

demonstrated

its

promise

of

being

a

self-adaptive

approach

for

autonomic

computing

in

virtualized

data

centers..

In

the

future

work,

we

will

continue

to

tackle

the

autonomic

performance

control

problem

from

the

following

three

perspectives.

First,

we

aim

to

enrich

the

neural

fuzzy

controller

by

developing

self-tuning

components

for

choosing

the

control

interval

and

other

key

parameters.

The

control

interval

choice

can

affect

performance

measurement

noise

and

responsiveness

of

control

actions.

We

want

to

extend

the

controller

to

support

multiple

heterogeneous

applications

with

varying

workload

characteristics.

Second,

we

will

consider

integrating

a

reinforcement-learning-based

component

for

resource

allocation

optimality.

Third,

we

want

to

integrate

admission

control

with

dynamic

server

allocation

when

the

resources

are

being

exhausted

in

the

system.

Autonomic

resource

management

for

performance

assurance

in

multi-tier

Internet

services

is

an

important

and

challenging

research

topic.

Recently,

there

are

a

few

studies

on

the

modeling

and

analysis

of

multi-tier

servers

with

queueing

foundations

[Bennani

and

Menasce

Diao

et

al.

Karve

et

al.

Liu

et

al.

Singh

et

al.

Stewart

et

al.

Urgaonkar

et

al.

Urgaonkar

et

al.

Diao

et

al.

described

a

performance

model

based

on

queueing

for

differentiated

services

of

multi-tier

applications.

Per-tier

concurrency

limits

and

cross-tier

interactions

were

addressed

in

the

model.

Villela

et

al.

studied

optimal

server

allocation

in

the

application

tier

that

increase

a

server

provider's

profits.

An

optimization

problem

is

constructed

in

the

context

of

a

set

of

application

servers

modeled

as

processor

sharing

queueing

systems.

Urgaonkar

et

al.

designed

a

dynamic

server

provisioning

technique

on

multi-tier

server

clusters

[Urgaonkar

et

al.

The

technique

decomposes

the

per-tier

average

delay

targets

to

be

certain

percentages

of

the

end-to-end

delay

constraint.

Based

on

a

queueing

model,

per-tier

server

provisioning

is

executed

at

once

for

the

per-tier

delay

guarantees.

There

is,

however,

no

guidance

about

the

decomposition

of

end-to-end

delay

to

per-tier

delay

targets.

Zhou

proposed

an

efficient

server

provisioning

approach

on

multi-tier

clusters

based

on

an

end-to-end

resource

allocation

optimization

based

on

a

queueing

model.

It

is

able

to

minimize

the

number

of

virtual

servers

allocated

to

the

system

and

satisfy

the

average

end-to-end

response

time

guarantee

under

stationary

workloads.

Singh

et

al.

proposed

a

novel

dynamic

provisioning

technique

that

handles

both

the

nonstationarity

in

the

workload

and

changes

in

request

volumes

when

allocating

server

capacity

in

data

centers.

It

is

based

on

the

k-means

clustering

algorithm

and

a

queuing

model

to

predict

the

server

capacity

for

a

given

workload

mix.

These

queueing-model-based

techniques

can

provide

the

average

response-time-based

performance

guarantee,

but

not

the

percentile-based..

Feedback

control

was

used

for

service

differentiation

and

performance

guarantee

on

Internet

servers

Jung

et

al.

Kamra

et

al.

Leite

et

al.

Liu

et

al.

Lu

et

al.

Padala

et

al.

Wei

and

Xu

Sha

et

al.

proposed

to

integrate,

queuing

model

with

feedback

control

for

average

response

time

control

of

Web

systems.

Kamra

et

al.

designed

a

Proportional

Integral

(PI)-controller-based

admission

control

proxy

to

bound

the

average

end-to-end

delay

in

a

three-tier

Web

service.

There

are

studies

that

argue

model-dependent

control

techniques

may

suffer

from

the

inaccuracy

of

modeling

dynamic

workloads

in

multi-tier

systems.

For

instance,

Lu

et

al.

modeled

a

controlled

Web

server

with

a

second-order

difference

equation

whose

parameters

were

identified

using

the

least

square

estimator.

The

estimated

system

model

used

for

control

would

become

inaccurate

if

the

real

workload

range

deviates

significantly

from

those

used

for

performance

model

estimation

[Lu

et

al.

Padala

et

al.

proposed

AutoControl,

a

combination

of

an

online

model

estimator

and

a

multi-input

multi-output

controller

[Padala

et

al.

The

resource

allocation

system

can

automatically

adapt

to

workload

changes

in

a

shared

virtualized

infrastructure

to

achieve

the

average

response-time-based

service-level

objectives.

The

approaches

can

provide

performance

guarantees

regarding

the

average

response,

but

not

percentile-based

response

time..

A

percentile-based

performance

metric

has

the

benefit

that

it

is

both

easy

to

reason

about

and

to

capture

individual

users'

perception

of

Internet

service

performance

[Lama

and

Zhou

Leite

et

al.

Urgaonkar

et

al.

Watson

et

al.

Welsh

and

Culler

Welsh

and

Culler

proposed

to

bound

the

th

-percentile

response

time

of

requests

in

an

multistage

Internet

server.

It

is

achieved

by

an

adaptive

admission

control

mechanism

that

controls

the

rate

of

request

admission.

The

mechanism

complements

but

does

not

apply

to

dynamic

server

provisioning

in

data

centers.

Urgaonkar

et

al.

proposed

an

interesting

approach

for

assuring

the

thpercentile

delay

guarantee.

It

uses

an

application

profiling

technique

to

determine

a

service

time

distribution

whose

th

-percentile

is

the

delay

bound.

The

mean

of

that

distribution

is

used

as

the

average

delay

bound.

It

then

applies

the

bound

for

the

pertier

delay

target

decomposition

and

per-tier

server

provisioning

based

on

a

queueing

model.

There

are

two

key

problems.

One

is

that

the

approach

is

queueing

model

dependent.

The

second

is

that

the

application

profiling

needs

to

be

done

offline

for

each

workload

before

the

server

replication

and

allocation.

Due

to

the

very

dynamic

nature

of

Internet

workloads,

application

profiling

itself

can

be

time

consuming

and

importantly

not

adaptive

online.

Lama

and

Zhou

proposed

a

fuzzy-control-based

server

provisioning

approach

that

can

effectively

bound

the

th

-percentile

response

time

of

an

Internet

service.

The

technique

works

well

under

stationary

system

workloads.

However,

it

does

not

effectively

adapt

to

the

very

dynamic

nature

of

Internet

workloads.

The

study

in

Singh

et

al.

found

that

the

nonstationarity

in

Internet

application

workloads

can

have

a

significant

impact

on

the

overall

processing

demands

imposed

on

data

center

servers.

Leite

et

al.

applied

an

innovative

stochastic

approximation

technique

to

estimate

the

tardiness

quantile

of

response

time

distribution,

and

coupled

it

with

a

Proportional-Integral-Derivative

(PID)

feedback

controller

to

obtain

the

CPU

frequency

for

single-tier

servers

that

will

maintain

performance

within

a

specified

deadline.

It

is

nontrivial

to

apply

this

approach

to

dynamic

server

allocation

problem

due

to

several

reasons.

First,

it

does

not

compensate

for

the

effect

of

process

delay

in

resource

allocation,

which

is

significant

due

to

server

switching

costs.

Second,

it

applies

a

PID

controller

without

using

any

system

performance

model

despite

the

fact

that

it

is

essentially

a

model-based

control

technique.

The

controller

was

designed

solely

based

on

response

time

measurement

and

manual

tuning

of

controller

parameters

for

a

particular

simulated

workload.

As

a

result,

it

may

not

be

adaptive

to

highly

dynamic

workloads.

Watson

et

al.

proposed

an

unique

approach

to

model

the

probability

distributions

of

response

time,

in

terms

of

percentiles,

based

on

CPU

allocations

on

virtual

machines.

The

performance

model

was

obtained

by

offline

training

based

on

data

collected

from

the

system.

It

is

not

adaptive

online

to

dynamically

changing

workloads.

The

work

focuses

on

performance

modeling

without

addressing

issues

related

to

adaptive

resource

provisioning

such

as

process

delay,

system

stability,

performance

assurance,

etc..

Fuzzy

theory

and

control

were

applied

for

Web

performance

guarantee

due

to

its

appealing

feature

of

model

independence,

and

used

to

model

uncertain

and

imprecise

information

in

applications

[Zhou

and

Huang

Liu

et

al.

used

fuzzy

control

to

determine

an

optimal

number

of

concurrent

child

processes

to

improve

the

Apache

Web

server

performance.

Wei

and

Xu

designed

a

fuzzy

controller

for

provisioning

guarantee

of

user-perceived

response

time

of

a

Web

page.

Those

fuzzy

controllers

were

designed

manually

on

a

trial-and-error

basis.

Important

design

parameters

such

as

input

scaling

factors,

rule

base,

and

membership

functions

are

not

adaptive.

They

are

not

very

effective

in

the

face

of

highly

dynamic

workloads.

In

this

article,

we

design

a

self-adaptive

neural

fuzzy

controller

which

is

capable

of

automatically

learning

its

structure

and

parameters

using

online

measurement..

Statistical

machine

learning

techniques

have

been

used

for

measuring

the

capacity

of

Web

sites

[Liu

et

al.

Rao

and

Xu

for

online

system

reconfiguration

[Bu

et

al.

and

for

resource

allocation

[Meng

et

al.

Tesauro

et

al.

Zhang

et

al.

For

instance,

Bu

et

al.

proposed

a

reinforcement

learning

approach

for

autonomic

configuration

and

reconfiguration

of

multi-tier

Web

systems..

In

this

article,

we

design

a

neural

fuzzy

controller

as

a

hybrid

of

control-theoretical

and

machine

learning

techniques

for

autonomic

server

allocation.

It

uses

an

online

learning

algorithm

to

self-construct

its

structure

and

adapt

its

parameters

based

on

live

incoming

data.

This

saves

a

significant

amount

of

time

and

efforts

required

to

collect

a

large

set

of

representative

training

data

and

to

train

the

system.

Furthermore,

it

avoids

poor

performance

of

a

typical

online

training

process

due

to

the

incorporation

of

feedback

control.

O O O O B I I O B I I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O


Quantification of Smoothing Requirement for 3D Optic Flow Calculation of Volumetric Images Abstract-Complexities of dynamic volumetric imaging challenge the available computer vision techniques on a number of different fronts. This paper examines the relationship between the estimation accuracy and required amount of smoothness for a general solution from a robust statistics perspective. We show that a (surprisingly) small amount of local smoothing is required to satisfy both the necessary and sufficient conditions for accurate optic flow estimation. This notion is called "just enough" smoothing, and its proper implementation has a profound effect on the preservation of local information in processing 3D dynamic scans. To demonstrate the effect of "just enough" smoothing, a robust 3D optic flow method with quantized local smoothing is presented, and the effect of local smoothing on the accuracy of motion estimation in dynamic lung CT images is examined using both synthetic and real image sequences with ground truth.Index Terms-3D optic flow, 4-D CT, Gaussian smoothing, volumetric images. A new approach to quantifying the minimum required smoothing based on the concept of the finite sample bias of a robust estimator is presented. The proposed approach is very general and makes predictions about required amount of smoothness to satisfy the sufficiency condition for a broad range of visual estimation tasks such as optic flow calculation. We particularly showed that smoothing over a cubic area as small as 5 to 7 voxels wide is sufficient to achieve the highest practical accuracy. This is a significant observation as it proves that very localized changes in motion in 3D data is directly observable. The predictions were tested for 3D optic flow estimation of 4D lung CT images and the results showed, inline with theoretical predictions, that only a very small amount of local smoothing is required to achieve high accuracy and observe local anatomically induced motion variations and that in some cases increasing the amount of smoothing deteriorates results.

O O O O O B I I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O