An integration of online and pseudo-online information for cursive word recognition Abstract-In this paper, we present a novel method to extract stroke order independent information from online data. This information, which we term pseudo-online, conveys relevant information on the offline representation of the word. Based on this information, a combination of classification decisions from online and pseudo-online cursive word recognizers is performed to improve the recognition of online cursive words. One of the most valuable aspects of this approach with respect to similar methods that combine online and offline classifiers for word recognition is that the pseudo-online representation is similar to the online signal and, hence, word recognition is based on a single engine. Results demonstrate that the pseudo-online representation is useful as the combination of classifiers perform better than those based solely on pure online information. O NLINE cursive word recognition is a challenging task ( [1] , [2] ). Currently, online recognition suffers from several weaknesses that involve sensitivity to stroke order, stroke number, and stroke characteristics variations ( [3] ). As similar shapes might be produced by different sets of strokes, two instances of the same letter or word may resemble each other in the image domain though they are associated with diverse online signals. This phenomenon is dominant at the beginning of words, after a pen-up pen-down motion and in the presence of letters that contain delayed strokes like i and t. Following the event of pen-up or, equivalently, at the beginning of a word, any writer may put the pen down in a different location and, hence, the next letter could be drawn in a unique manner. Delayed strokes may appear at any time and, consequentially, at any place, within the online signal. Fig. 1 presents two instances of the letter "o" that have the same topology-a single loop with no tails-and similar geometry-mostly concave with global extreme points at each side. Nevertheless, since each one of these letters was drawn from a different starting point (the beginning of stroke number 1), it resulted in a unique set of strokes. Another side of this phenomenon is observed in letters like b, d, h, i, and l, where an ascender that looks like a thick pole can be made by either one of an elongated loop or cusp form.. A special group of strokes that often cause inconsistency problems are the ones we term redundant strokes. There are many superfluous pixels that are byproducts of the continuous nature of cursive handwriting. On the one hand, such redundant strokes do not appear constantly because some writers use the pen-up pen-down option instead. On the other hand, in the time domain, adjacent strokes remain disjoint even if they meant to overlap, so it is only in the image domain that they coincide. See, for example, Fig. 2 where there is zoom-in on two instances of the letter "a." The difference between the two samples is the ligature that connects the letter in Fig. 2b to its preceding (strokes number 1 and 2). Another aspect of redundant strokes that is very common in real scenes is strokes that are repeated either intentionally or accidentally. Sometimes, the writer goes back and corrects himself. There are other times when he mistakenly produces a slip of "ink" at the end of a letter.. In a similar way, incomplete (open) loops are more frequent in the online representation. It seems that many writers count on stroke width and smearing effects to close the loops they sketch. Hence, it would be only in the real offline representation (a scanned bitmap image) that some loops mature. This phenomenon, which is demonstrated in Fig. 2a , is also considered a major problem in online recognition.. In this paper, we present a novel approach to extract an alternative list of strokes from an online signal. The resulting representation, which we term pseudo-online, is a two-dimensional signal of pixel locations along an imaginary uniformly distributed time axis-ðxðtÞ; yðtÞÞ.. Our aim is to provide a method that produces a new representation for a given word in which the abovementioned phenomena will be minor. In order to achieve this goal, we suggest that one should refer to the associated bitmap image of the online word with the purpose of deriving the proposed alternative list of strokes. To generate this bitmap image, we simulate the image that would have been captured by scanning a real scene. The main emphasis is on giving strokes a significant width.. The main advantage of the bitmap image representation is the resemblance between instances of the same word. In this case, the deterministic algorithm that we describe next would derive a consistent list of strokes from the bitmap image of the word. In addition, many genuine strokes are filtered out because of the blurring effect and the lack of temporal information. This process reduces the number of redundant strokes. Using this approach, we propose a new strategy to bootstrap an alternative list of strokes, the pseudo-online, from an online signal, using the bitmap image representation as a buffer. The idea is to regenerate an ordered list of strokes that would represent the input word in a different manner, mostly in view of spatial closeness between adjacent strokes. Thus, we produce new offline-like features that support an improved online recognition process.. Several methods were proposed to do an offline to online transformation. They were usually referred to as temporal information recovery ( [10] , [19] , [20] ). Nonetheless, in contrast with these methods, our algorithm does not aim to trace the original path of the pen. This would have been less useful for the purpose of integration with the genuine online signal because there would have been no diversity between their outputs. Instead, we control the pseudo-online transformation to be complementary to the online signal and create true additional recognition critical information.. In order to serve the concept of a transformation that keeps resemblance between images, the information carried by the pseudo-online representation is analogous to the shapes of the characters creating the word. This is achieved by referring to the external contour of significant parts like ascenders, descenders, and loops. Furthermore, the image is always traversed from the leftmost pixel to the rightmost pixel and, in this way, a consistent order among the branching parts is enforced. For similar reasons, all connected components, with the exception of i-dots, are concatenated. The latter are virtually associated with the closest peak in the middle zone of the word.. The proposed pseudo-online representation enables one to recognize words that are not recognized easily given the genuine online signal. However, there are many other words for which recognition could be easier using the original signal alone. Usually, the pseudo-online representation would be inferior when valuable strokes disappear in the bitmap image. Sometimes, temporal information is also a must. This is the case, for example, when stroke order is the only way to distinguish between similar objects ([4], [5] ).. In general, online classifiers are superior to pure pseudoonline classifiers. When one exploits offline-like features derived from bitmap images, he might lose valuable significant informative strokes that are filtered out ( [4] , [5] ). In a similar way, offline classifiers are also limited to some extent ( [6] , [7] , [8] ).. As a result, one cannot replace the online representation with the pseudo-online. Nevertheless, in order to get closer to optimality, we suggest using them both. We propose working with a combination of classifiers trained individually on either online or pseudo-online representations, respectively. Thus, the heterogeneous classifiers would compensate each other. The transformation from online to pseudo-online via bitmap image should be configured to compensate and support the pure online representation, creating new features and, hence, new recognition ability. This will increase the group of recognizable words span by the combination of classifiers and maximize the overall recognition rate. Fig. 1 . Two instances of the letter "o" that were produced by diverse sets of strokes due to a different starting point, but resulted in similar shapes, i.e., the same topology-a single loop with no tails-and similar geometry-mostly concave with global extreme points at each side. Fig. 2 . Two instances of the letter "a" that resemble each other in the image domain but present diverse sets of strokes. While the first sample starts with a pen-down operation, the second one is connected by a ligature to the preceding (strokes number 1 and 2). This ligature is hidden in the bitmap image because it was overlapped by the other strokes. The first sample also demonstrates a loop that was left open (strokes number 1 and 3 are not in touch), but became closed in the offline representation (in white).. The fact that the pseudo-online is also in the form of pixels (strokes) as a function of (an imaginary) time is the key to a dual practice of a single online word recognition engine.. Therefore, one can utilize a single online word recognition engine to run both online and pseudo-online data. In this case, there will be two sets of trained instances of the same engine-for the genuine and pseudo-online signals, respectively. Both sets of classifiers are trained on the same data and the dissimilarity is gained from the unique extended preprocessing.. We propose runing online word recognition in two parallel paths-see Fig. 3 . The given input word is duplicated and processed simultaneously by online and pseudo-online classifiers. In the pseudo-online root, a preprocessing stage precedes the recognition. It consists of three steps: First, the online signal is projected to the image domain, then consecutive pixels are connected by thick lines to establish meaningful stroke width, and, eventually, the pseudo-online signal is extracted. All the classifiers are instances of a single online word recognition engine. The outputs of all classifiers from both paths are collected and fused by a common combination scheme.. There were previous methods that suggested a combination of classifiers for online recognition ( [9] , [10] ). Some of them proposed embedding offline-like data in the classical stroke representation during preprocessing ( [11] , [12] , [13] , [14] ). Other methods suggested integration with pure offline word recognition systems at the postprocessing stage ( [15] , [16] , [17] , [18] ). We propose using the same online word recognition engine to process both types of data. Thus, maintenance is simplified to a single engine with a single data set. Moreover, according to Velek et al. ([18] ), who investigated online and offline classifier combinations, it requires some relation among the confidence values that are attached to candidates stemming from different classifiers in order to compare them. Sometimes, various normalization functions are necessary to overcome their inadequacies. Therefore, an integration of classifiers based on the same recognition engine is more straightforward.. Our experiments show that such a mixed combination of classifiers improves the recognition rate suggested by a combination of online classifiers solely. In general, online classifiers are not sensitive to the same words that the pseudoonline classifiers are. These results also demonstrate that the Fig. 3 . Data flow in the proposed method. The online signal is duplicated and processed simultaneously by online and pseudo-online classifiers. The pseudo-online root presents some preprocessing stages that precede the recognition: projection to the image domain, drawing of thick lines between consecutive pixels, and the pseudo-online transformation. Both representations are then fed to several classifiers that are instances of the same online word recognition engine. The output results from both paths are collected and fused by a common combination scheme.. features conveyed by the pseudo-online representation are informative and useful.. The paper is organized as follows: Section 2 describes the proposed pseudo-online transformation. Section 3 explains the integration of online and pseudo-online classifiers. Section 4 presents experimental results that are analyzed and discussed in Section 5. We summarize with some insights and conclusions in Section 6. The recognition rates that were achieved by the mixed combinations of online and pseudo-online classifiers improve previous results that were accomplished on this data set ( [24] , [25] , [29] ). We have demonstrated that the pseudoonline representation is useful and provides additional information that can be easily translated into significant recognition rate improvement. We have shown that the variability in the recognition rate improvement is reduced when more classifiers (with different letter neighborhood sizes) are used. In the latter case, the number of pseudoonline classifiers that were part of the optimal combination was increased, emphasizing the important contribution of the pseudo-online representation to the genuine online signal. Moreover, it proves that the improvement was orthogonal to the improvement that could be obtained by introducing only new online classifiers. This is further supported by the fact that the significance of the improvement was not changed when combining 12 classifiers, while the absolute recognition rate has jumped from 76.5 percent to 82.3 percent (in the case of the sum rule combination scheme). The upper bound recognition rate, which indicates the best performance that could be achieved from the collection of classifiers at hand (while not indicating a specific strategy for achieving such combination), retains the significance in the improvement while indicating far better absolute recognition results. For example, in the case of 12 classifiers, the upper bound recognition rate is over 92.9 percent compared to 85.3 percent in the case of 6-classifier combination. As an optimal combination scheme is outside the scope of this paper, this result is sufficient to indicate that the pseudo-online representation does add information that cannot be obtained by optimizing a combination of online classifiers only. Fig. 18 . (a) Number of input words that were recognized by at least n more pseudo-online classifiers than online classifiers ð1 n 12Þ. The histogram summarizes recognition results on a data set of 18 writers Â 862 word classes. (b) Number of input words that were recognized by at least n pseudo-online classifiers and neither one of the online classifiers ð1 n 12Þ for the above-mentioned data set. Fig. 19 . Two samples of input words that were recognized correctly by at least 10 of the 12 pseudo-online classifiers and neither one of the 12 online classifiers. The word "lead" in (a) presents an open loop at the "a" and a ligature that interferes with the loop of the "d." Both problems were overcome in the bitmap image and the pseudo-online representation that was derived from it. The word "science" in (b) presents a delayed i-dot that is not positioned within the main body and a "c" that resembles an "e." In the latter case, this form of "c" is more likely to be recognized by pseudo-online classifiers because of the tarsus representation that was demonstrated in Section 2. The resolution of the dissimilarities between the online and pseudo-online signals is at the letter level. There was some evidence that pseudo-online classifiers prefer words with complicated letters like f and k or words that begin with the letters a,b, h, i, and l. However, in general, the association between the input words that were better recognized by the pseudo-online classifiers and specific letters was not conclusive. This is likely due to the fact that the recognition engine averages the confidences of all letters in a word and masks local peaks. Nevertheless patterns of improvement could be seen at the word as well as the writer levels. The collection of results obtained in Experiment 4 shows that there were 110 word classes (12.8 percent) in which at least seven word samples (10.6 percent) were correctly recognized only by the combination with the pseudo-online classifiers. The ordered histogram presented in Fig. 17a summarizes, for each word class, the percentage of word samples that were correctly recognized only by the mixed combination of classifiers.. At the writer level, we observed that there were 12 writers (18.2 percent) for whom at least 65 of the words they produced (7.5 percent) were correctly recognized only by the combination with the pseudo-online classifiers. More than a 12 percent difference was obtained for the top two writers. The ordered histogram that summarizes the percent differences for all the writers is given in Fig. 17b .. Following the analysis of the mixed combination performance, we did some investigation to learn about the correlation among the pseudo-online classifiers in comparison with the correlation between online and pseudo-online classifiers, respectively. Let O i and P O i be the number online and pseudo-online classifiers that correctly recognized the ith input word, respectively, then the histogram of all possible positive values of P O i À O i ð½1; . . . ; 12Þ that were observed in the 18-writer test set ð1 i 18 Ã 862 ¼ 15; 516Þ is given in Fig. 18a . Similarly, the distribution of the input words for which P O i ! nð1 n 12Þ and O i ¼ 0 is shown in Fig. 18b . The statistics demonstrate that the pseudo-online classifiers share unique characteristics that reflect on a subdomain of recognizable input words. For example, there were 909 input words (5.9 percent) that were correctly recognized by at least one pseudo-online classifier and neither one of the 12 online classifiers. Fig. 19 presents two of the leading words in this group that were recognized by at least 10 of the 12 pseudo-online classifiers.. To complete the picture, we have computed the mean diversity of the errors between a pair of online, a pair of pseudo-online and an online plus a pseudo-online pair of classifiers, respectively. Let Cl i ðjÞ be the word class to which the input word j was classifieds by classifier i, then the diversity of the errors made by classifiers i 1 ; i 2 is given by: The results show correspondence among the various online and pseudo-online classifiers-86:4% AE 6:1 and 80:0% AE 5:9, respectively. This is the antithesis of the high diversity that was found between pairs of online and pseudo-online classifiers-94:3% AE 3:3.

O O O B O O O O B B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O B O O O O O O O O O B B O O O O O O O O O O O O O O O O O B O B O O B I O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O B B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O


Reinforcement Learning: A Tutorial Survey and Recent Advances In the last few years, Reinforcement Learning (RL), also called adaptive (or approximate) dynamic programming (ADP), has emerged as a powerful tool for solving complex sequential decision-making problems in control theory. Although seminal research in this area was performed in the artificial intelligence (AI) community, more recently, it has attracted the attention of optimization theorists because of several noteworthy success stories from operations management. It is on large-scale and complex problems of dynamic optimization, in particular the Markov decision problem (MDP) and its variants, that the power of RL becomes more obvious. It has been known for many years that on large-scale MDPs, the curse of dimensionality and the curse of modeling render classical dynamic programming (DP) ineffective. The excitement in RL stems from its direct attack on these curses, allowing it to solve problems that were considered intractable, via classical DP, in the past. The success of RL is due to its strong mathematical roots in the principles of DP, Monte Carlo simulation, function approximation, and AI. Topics treated in some detail in this survey are: Temporal differences, Q-Learning, semi-MDPs and stochastic games. Several recent advances in RL, e.g., policy gradients and hierarchical RL, are covered along with references. Pointers to numerous examples of applications are provided. This overview is aimed at uncovering the mathematical roots of this science, so that readers gain a clear understanding of the core concepts and are able to use them in their own research. The survey points to more than 100 references from the literature. A. Gosavi Markov decision problems (MDPs) are problems of sequential decision-making in which a control (action) has to be selected in each decision-making state visited by the concerned system. Such problems are widespread in stochastic control theory, and their roots can be traced to the pioneering work of Richard Bellman in the fifties. The main contribution of Bellman's work was to show that the computational burden of an MDP could be dramatically reduced via, what is now well-known as, dynamic programming (DP). However, it was also recognized quite early in the historical evolution of this problem domain that on large-scale and complex MDPs, methods of classical DP, namely policy iteration (PI) and value iteration (VI), break down. The requirement of computing, storing, and manipulating the so-called transition probability matrices (TPMs) is responsible for this breakdown in classical DP. In problems involving complex systems with several governing random variables, it is usually difficult to compute the values of the transition probabilities (TPs). This phenomenon is called the curse of modeling. In problems with a large dimension, storing or manipulating the elements of the so-called value function -needed in DP -becomes challenging. This is called the curse of dimensionality. As such, classical DP, even today, is rather ineffective on large-scale and/or complex problems.. The power of Reinforcement Learning (RL) or Adaptive (or approximate) DP (ADP) lies in its ability to solve, near-optimally, complex and large-scale MDPs on which classical DP breaks down. RL emerged as a tool in the artificial intelligence (AI) and neural research communities, where combining DP with derivative-based adaptive function approximations (Werbös, 1987) and learning-based methods (Barto et al., 1983) was advocated in the mid-1980s. The modern science of RL has emerged from a synthesis of notions from four different fields: classical DP, AI (temporal differences), stochastic approximation (simulation), and function approximation (regression, Bellman error, and neural networks). In this survey, we will discuss the main ideas in RL with special attention to the underlying mathematical principles. We will describe a few important algorithms, along with pointers to some case studies. Outside the MDP, we will also present some recent advances in solving other problems such as semi-Markov decision problems (SMDPs), competitive MDPs (also called stochastic games), and hierarchical MDPs. The survey points to more than 100 references from the existing literature, and it is hoped that new ideas for research will be stimulated from reading this paper.. The rest of this article is organized as follows. Section 2 introduces the MDP framework. Section 3 presents an overview of DP-based RL conducted in a simulation environment. Extensions of RL to other domains, along with recent developments and some applications, are presented in Section 4. Section 5 concludes this survey, with pointers to active areas of current research. The MDP (Bellman, 1954; Howard, 1960) has found numerous applications in industrial settings (White, 1985) . However, the curses of modeling and dimensionality have plagued it for many years. Because of these curses, DP has not been very effective in solving many realistic MDPs in the real world. The advent of RL has engendered a significant change in our ability to solve MDPs -especially those that have complex transition mechanisms and large state spaces.. RL is unique in its use of the combination of four distinct machineries: stochastic approximation, DP, AI and function approximation. By exploiting these machineries, RL has opened the avenue for solving large-scale MDPs (and its variants) in discrete-event systems, which were considered intractable in the past. RL has outperformed industrial heuristics in several of the published case studies of industrial importance. RL as a science is relatively young and has already made a considerable impact on operations research. The optimism expressed about RL in the early surveys (Keerthi and Ravindran, 1994; Kaelbling et al., 1996; Mahadevan, 1996) has been bolstered by several success stories.. Although a great deal of work in algorithmic development of RL has already occurred, RL continues to attract research attention. In order to familiarize the reader to open problems in RL, we enumerate some areas undergoing active research. Function approximation (Mahadevan and Maggioni, 2006; Whiteson and Stone, 2006) to this day remains an unsolved problem because of the lack of stability of known approximators (discussed in Section 3.3). Hierarchical RL (Bhatnagar and Panigrahi, 2006 ) also promises to solve large-scale problems and those with special multi-layer structures. Policy gradients (Singh et al., 2005) has emerged as a new area in RL research with a gradient-descent approach to the MDP. Interest in Linear programming (LP) for solving MDPs has been renewed (Defarias and van Roy, 2003) because of the well-known stability properties of LP solvers. Other areas that are attracting interest are: risk-sensitive RL (Borkar, 2002; Geibel and Wysotzki, 2005) , factored MDPs (Schuurmans and Patrascu, 2002) , and analyzing computational complexity (Evan-Dar and Mansour, 2003) .

B I O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Discovering frequent user--environment interactions in intelligent environments Abstract Intelligent Environments are expected to act proactively, anticipating the user's needs and preferences. To do that, the environment must somehow obtain knowledge of those need and preferences, but unlike current computing systems, in Intelligent Environments, the user ideally should be released from the burden of providing information or programming any device as much as possible. Therefore, automated learning of a user's most common behaviors becomes an important step towards allowing an environment to provide highly personalized services. In this article, we present a system that takes information collected by sensors as a starting point and then discovers frequent relationships between actions carried out by the user. The algorithm developed to discover such patterns is supported by a language to represent those patterns and a system of interaction that provides the user the option to fine tune their preferences in a natural way, just by speaking to the system. Ubiquitous Computing, a term introduced by Mark Weiser [36] , refers to a paradigm, in which a new type of relation between users and technology is established such that technology is both widespread and transparent to the user. Now computing devices of various types are all around us, embedded in different objects we interact with, and in that way they influence our lives. An important further development of this concept has resulted in concepts such as Ambient Intelligence (AmI) [1, 15, 20, 31] or Intelligent Environments [10] . Other terms, such as Smart Environments [16] or Pervasive Computing [22] , are used with similar connotations. All of them refer to digital environments that proactively, but sensibly, support people in their daily lives [5] . As mentioned by Augusto [6] , ''In order to be sensible, a system has to be intelligent, that is, how a trained assistant, e.g., a nurse, typically behaves. It will help when needed, but will restrain to intervene unless is necessary. Being sensible demands recognizing the user, learning or knowing her/his preferences, and the capability to exhibit empathy with the user's mood and current overall situation''. In Intelligent Environments, learning means that the environment must gain knowledge about the preferences, needs, and habits of the user in order to be in a better position to assist the user adequately [24] .. Let us consider the following scenario, which exemplifies the common behavior of a user. On weekdays Michael's alarm clock goes off (''Alarm on'') few minutes after 08:00AM. Approximately 10-15 min after getting up he usually steps into the bathroom (''Bathroom on'') and (2 s after) he turns on the lights of the bathroom (''BathLights on'') if the bathroom is dark (bathroom light level\ 10). On Tuesdays, Thursdays and Fridays he usually takes a shower (''Shower on''; Michael prefers the temperature of the water to be around 24-26 degrees Celsius in the winter and around 21-23 degrees Celsius in the summer. When he finishes taking a shower and 10 s after he turns on the fan of the bathroom (''BathFan on''), only when the relative humidity level is [ 75%. Before he leaves the bathroom (''Bathroom off'') he turns off the fan (''BathFan off'') and the lights (''BathLights off''). Intelligent Environments need to know the common behaviors and preferences of their users in order to meaningfully assist them. We have developed a system called PUBS that aims precisely at supporting an Intelligent Environment in the important task of understanding what the frequent behaviors of the occupant are in a given environment. This supports decision-making that can help the user, and it is also flexible enough to track the different behaviors we humans exhibit at different times.. PUBS was developed taking into account all of the specific characteristics of Intelligent Environments. Thus, both the acquisition process and the feedback process are carried out as unobtrusively as possible. In that sense, PUBS integrates an interaction system (I PUBS ) based on speech that facilitates interaction with the user.. The experiments carried out to validate PUBS have shown the capacity of PUBS to discover relationships, time relations, and the conditions of such relationships. These experiments (especially the one carried out using the WSU data set) allowed us to realize that common behaviors of the user would be better represented by means of sequences of actions instead of one-to-one relationships. Such representation facilitates understanding behaviors, and they may be used for other purposes, such as the automation of actions. Thus, our current work is focused on discovering Learning is an essential feature in any Intelligent Environment. However, given the diversity of elements that must converge in order to obtain the infrastructure needed for an Intelligent Environment, learning has not received as much attention in the literature as it may require. Some notable exceptions are mentioned below.. Mozer et al. [29] and Chan et al. [12] were among the first reports on applications for AmI environments, in which user patterns were considered. In the case of Mozer et al., the aim of their system (installed in the Adaptive House) was to design an adaptive control system that considers the lifestyle and energy consumption of the inhabitants. For that, they used a feed-forward neural network to develop an occupancy predictor and zone anticipator, which were used to predict where the user would be in the coming seconds and control lighting based on those predictions. Chan et al. developed a similar application in order to asses whether a given situation was normal or abnormal. Other authors have kept using ANNs in order to provide personalized services. Thus, Campo et al. [11] developed a system that calculated the probability of occupation for each area of the house and systematically compared the probability with the current situation. A survey of those works can be found in [1] . The use of ANNs has a limitation related to their black box nature; therefore, their internal structure is not comprehensible.. The group that has been working on the MavHome and Casas projects is one of the most active groups in this area. The first applications developed by this group were focused on building universal models, represented by means of Markov models, in order to predict either future locations or activities [17] . In this sense, they made notable improvements by developing applications to discover daily and weekly patterns [26] . Additionally, they constructed applications with the ability to infer abstract tasks automatically, identifying corresponding activities that were likely to be part of the same task [32] . Jakkula and Cook [27] extended this work to predict actions using temporal relations, defined by means of Allen's [4] temporal logic relations. They have also developed applications to recognize activities [28] .. Researchers at Essexs iDorm laboratory have given prominence to the problem of learning and are one of the most active groups in this area. Their initial efforts [18, 25] were focused on developing an application that generated a set of fuzzy rules to represent users patterns. Recording the changes caused by the user in the environment, they generated membership functions as well as fuzzy rules that mapped data into fuzzy rules. Due to the excessive number of rules they generated, they introduced an improvement [21] where they identified relevant and important associations between given actions, so that irrelevant aspects of the rules (and, by extension, some rules as well) were removed.. Other techniques have also been used. The group that works in the environment named SmartOffice [23] developed an algorithm that discovered conditions for situations in which examples indicate different reactions. Research conducted under the MyCampus project filtered messages based on the preferences of the user [34] . Including a system, based on CBR, they significantly improved the quality of filtering (i.e., user satisfaction increased from 50 to 80%).. A survey of all these works can be found in [8] . We can state that due to specific characteristics of Intelligent Environments, each problem calls for the use of certain techniques, but as Muller [30] pointed out, ''the overall dilemma remains: there does not seem to be a system that learns quickly, is highly accurate, is nearly domain independent, does this from few examples with literally no bias, and delivers a user model that is understandable and contains breaking news about the user's characteristics''.

O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Language-specific models in multilingual topic tracking Topic tracking is complicated when the stories in the stream occur in multiple languages. Typically, researchers have trained only English topic models because the training stories have been provided in English. In tracking, non-English test stories are then machine translated into English to compare them with the topic models. We propose a native language hypothesis stating that comparisons would be more effective in the original language of the story. We first test and support the hypothesis for story link detection. For topic tracking the hypothesis implies that it should be preferable to build separate language-specific topic models for each language in the stream. We compare different methods of incrementally building such native language topic models. Topic detection and tracking (TDT) is a research area concerned with organizing a multilingual stream of news broadcasts as it arrives over time. TDT investigations sponsored by the U.S. government include five different tasks: story link detection, clustering (topic detection), topic tracking, new event (first story) detection, and story segmentation. The present research focuses on topic tracking, which is similar to filtering in information retrieval. Topics are defined by a small number of (training) stories, typically one to four, and the task is to find all the stories on those topics in the incoming stream.. TDT evaluations have included stories in multiple languages since 1999. TDT2 contained stories in English and Mandarin. TDT3 and TDT4 included English, Mandarin, and Arabic. Machinetranslations into English for all non-English stories were provided, allowing participants to ignore issues of story translation.. All TDT tasks have at their core a comparison of two text models. In story link detection, the simplest case, the comparison is between pairs of stories, to decide whether given pairs of stories are on the same topic or not. In topic tracking, the comparison is between a story and a topic, which is often represented as a centroid of story vectors, or as a language model covering several stories.. Our focus in this research was to explore the best ways to compare stories and topics when stories are in multiple languages. We began with the hypothesis that if two stories originated in the same language, it would be best to compare them in that language, rather than translating them both into another language for comparison. This simple assertion, which we call the native language hypothesis, is easily tested in the TDT story link detection task.. The picture gets more complex in a task like topic tracking, which begins with a small number of training stories (in English) to define each topic. New stories from a stream must be placed into these topics. The streamed stories originate in different languages, but are also available in English translation. The translations have been performed automatically by machine translation algorithms, and are inferior to manual translations. At the beginning of the stream, native language comparisons cannot be performed because there are no native language topic models (other than English). However, later in the stream, once non-English documents have been seen, one can base subsequent tracking on native-language comparisons, by adaptively training models for additional languages. There are many ways this adaptation could be performed, and we suspect that it is crucial for the first few non-English stories to be placed into topics correctly, to avoid building non-English models from off-topic stories.. Previous research in multilingual TDT has not attempted to compare the building of multiple language-specific models with single-language topic models, or to obtain native-language models through adaptation. The focus of most multilingual work in TDT for example [2] [12] [13] , has been to compare the efficacy of machine translation of test stories into a base language, with other means of translation. Although these researchers normalize scores for the source language, all story comparisons are done within the base language. This is also true in multilingual filtering, which is a similar task [14] .. The present research is an exploration of the native language hypothesis for multilingual topic tracking. We first present results on story link detection, to support the native language hypothesis in a simple, understandable task. Then we present experiments that test the hypothesis in the topic tracking task. Finally we consider several different ways to adapt topic models to allow native language comparisons downstream.. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Although these experiments were carried out in service of TDT, the results should equally apply to other domains which require the comparison of documents in different languages, particularly filtering, text classification and clustering. We have confirmed the native language hypothesis for story link detection. For topic tracking, the picture is more complicated. When native language training stories are available, good native language topic models can be built for tracking stories in their original language. Smoothing the native models with global models improves performance slightly. However, if training stories are not available in the different languages, it is difficult to form native models by adaptation or by translation of training stories, which perform better than the adapted global models.. Why should language specific comparisons be more accurate than comparisons based on machine translation? Machine translations are not always good translations. If the translation distorts the meaning of the original story, it is unlikely to be similar to the topic model, particularly if proper names are incorrect, or spelled differently in the machine translations than they are in the English training stories, a common problem in English translations from Mandarin or Arabic. Secondly, even if the translations are correct, the choice of words, and hence the language models, are likely to be different across languages. The second problem could be handled by normalizing for source language, as in [12] . But normalization cannot compensate for poor translation.. We were surprised that translating the training stories into Arabic to make Arabic topic models did not improve tracking, but again, our dictionary based translations of the topic models were different from native Arabic stories. We intend to try the same experiment with manual translations of the training stories into Arabic and Mandarin. We are also planning to investigate the best way to normalize scores for different languages. When TDT4 relevance judgments are available we intend to replicate some of these experiments on TDT4 data. Experiments are replicated with two different data sets, TDT3 and TDT4, and two very different similarity functions -cosine similarity, and another based on relevance modeling, described in the following two sections. Cosine similarity can be seen as a basic default approach, which performs adequately, and relevance modeling is a state of the art approach which yields top-rated performance. Confirming the native-language hypothesis in both systems would show its generality.. In the rest of this section, we describe the TDT data sets, then we describe how story link detection and topic tracking are carried out in cosine similarity and relevance modeling systems. Next, we describe the multilingual aspects of the systems.

O O O B B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O


Mobile phone based AR scene assembly In this paper we describe a mobile phone based Augmented Reality application for 3D scene assembly. Augmented Reality on mobile phones extends the interaction capabilities on such handheld devices. It adds a 6 DOF isomorphic interaction technique for manipulating 3D content. We give details of an application that we believe to be the first where 3D content can be manipulated using both the movement of a camera tracked mobile phone and a traditional button interface as input for transformations. By centering the scene in a tangible marker space in front of the phone we provide a mean for bimanual interaction. We describe the implementation, the interaction techniques we have developed and initial user response to trying the application. Unlike desktop computer applications, input options on mobile phones are very restricted. Current mobile phone interfaces are dominated by keypad input. Although limited, it is convenient to use the dial buttons for accessing menus and controlling applications. For improved navigation control, the keypad has been extended by five-way joypads or joysticks. In addition, some high-end smartphones feature stylus input similar to PDAs. Another interface technique is speech recognition e.g. for selecting which contact to call.. None of these techniques are suited for interaction with 3D objects or graphical scenes. In this case both the user viewpoint and individual object position and orientation need to be set interactively. In recent years the range of interface techniques with desktop computer interfaces has been extended with motion sensors and trackers. These allow the user to make inputs by moving the device itself. For example, in the BAT interface [31] virtual objects can be manipulated in 3D space by moving wooden handles that have a magnetic tracker embedded inside them. Instead of pressing a mouse button the user moves the BAT handle naturally and this is mapped into virtual object translation and rotation. Using a range of sensors, desktop interfaces can be developed that provide intuitive six degree of freedom (DOF) input for 3D applications. However there has been no such work presented on the handheld or mobile phone platform.. New opportunities in mobile phone interaction have emerged with the integration of cameras into the phones. By analyzing the video stream captured by the camera, using simple image processing on the phone, it is possible to estimate the movement of the device. This can be used in a number of ways such as providing a 6 DOF interface, or recognizing objects to make the phone context aware. In this paper we provide the first example of using phone motion to manipulate graphical objects in 6 DOF to create virtual scenes.. Our work also uses Augmented Reality display technology. Augmented Reality (AR) [33] is a technique for visualizing virtual information where the physical reality is part of the scene. The virtual information is registered in three dimensions with the real world and is rendered with a different projection depending on the position and orientation of the viewing device. The main advantage of AR is the elimination of context switching between the real and virtual domains. Another advantage is that real world reference points can aid the navigation of virtual content. AR applications typically use desktop computers. It is only recently that such applications have begun to appear on mobile phones, and ours is the fix example of an AR scene assembly application on a phone.. Traditionally the way to display AR content was to view it through a head mounted display (HMD). Wearing a HMD leaves the users hands free to interact with the virtual content, either directly or through an input device such as a mouse or digital glove. One disadvantage with the HMD is that a 2D GUI is likely to block the user's view of the real world. An alternative approach is handheld AR where the display is non-obscuring and can be compared to a looking glass. For handheld AR the user looks through the screen of the device to view the AR scene and needs at least one hand to hold the device. The user interface for these applications is very different than those for HMD based AR applications.. In recent years AR applications have migrated to a variety of handheld platforms, including Tablet PCs [27] , PDAs [28] and mobile phones [11] . Mobile phones are an ideal platform for AR thanks to the integrated camera that allows high quality optical tracking. In addition to integrated cameras the current generation of phones have full color displays, fast processors and even dedicated 3D graphics chips. Henrysson [11] and Moehring [16] have shown how mobile phones can be used for simple single user AR applications. In their work they create custom computer vision libraries that allows developers to build video see through AR applications that run on a mobile phone. Henrysson [9] has also researched face-to-face collaborative AR on mobile phones, finding that users prefer to collaborate together in an AR application than a purely graphical application on the phone.. In this paper we present an application that partly builds upon the conclusions of our previous user study [10] conducted in order to compare different interaction techniques for virtual object manipulation. We show how different strategies can be combined for manipulation of a general 3D scene using a standard mobile phone.. In the next section we review related work on handheld AR, camera based mobile HCI and 3D interaction on mobile phones. Next we discuss the relevant user interface aspects of the mobile phone. We then describe our platform and the scene assembly application in detail, and initial user feedback to the application. Finally we provide some directions for future research. The present platform allows a single user to manipulate position and orientation of 3D objects. In future versions we will extend this to allow collaboration between multiple users sending scene graph updates via Bluetooth. We also want to go beyond treating the objects as rigid and let the user edit the geometry. To do this we need to come up with a way to select individual vertices, edges and polygons. A rigorous user study with different tasks needs to be done.. We have presented an application that represents a new way of 3D interaction using mobile phones. We have built a general scene assembly application using an optimized port of ARToolKit. The application allows both isomorphic (keypad) and isometric (tangible) 6 DOF manipulations while allowing interactive viewing. The current restriction is the tracking range of the marker based optical tracking. This is the first AR scene assembly program developed for the mobile phone. The lessons that we have learned developing this application and the feedback from users will allow us to develop further AR modeling and graphics applications for the mobile phone environment. Our initial user experiences indicate that our set-up allows 6 DOF manipulation for scene assembly applications. By using an easily accessible menu we can map keys to axis instead of functions. Thus we can extend the interface to other operations such as scaling, cloning and various object specific features.. We have not conducted any formal usability studies yet, but a handful of people have tried the interface informally. The majority felt that they were able to manipulate the objects as intended. However many seemed to release the joypad button immediately on selection thinking that the object would still be attached to the phone. We believe that toggling selection in tangible mode by pressing the joypad button would introduce a source of error at the moment of release due to muscle contractions in the users' hand. This can be noticed slightly in the keypad mode when the object to be manipulated is selected. If the joypad button is not released immediately the selected object is likely to move due to fine motions of the users' hand. This is a common interface problem where selection and translation are performed in similar ways.. The main limitation is the tracking as the square must be visible at all times. We use multiple markers to extend the tracking range. This adds complexity to the calculations but we have managed to solve the associated problems. We have also experimented with motion flow tracking to allow one corner of the square marker to be outside the image, but this needs more work.. We believe our work can serve as a base for tabletop 3D applications where the spatial relationship between the objects is important. We assume most such applications will be games similar to the described virtual LEGO® example, but some Virtual Reality applications that require 6 DOF could possibly be ported.. In fact there are currently few interaction devices that offers true 6 DOF input and most of these are used in advanced Virtual Reality environment requiring tracking. Our solution could provide an inexpensive 3D mouse solution for PCs by sending the position and orientation information via Bluetooth to the computer. Since the tracking is performed on the phone there is no computational overhead for the PC compared to a 2D mouse. There are several examples of camera-based interaction with mobile devices. Two of the best known are "Mosquito Hunt" [18] and "Marble Revolution" [15] . In "Mosquito Hunt", virtual mosquitoes are superimposed over a live video image from the camera and simple motion flow techniques are used to allow the user to shoot the Mosquitos by moving the phone. Similarly, in the "Marble Revolution" game the player can steer a marble through a maze by moving the phone and using motion flow techniques. Neither have 3D registration of the graphics overlaid on the real world.. The virtual soccer game of KickReal [12] allows people to see a virtual ball superimposed over video of the real world and kick it with their feet, but there is no 3D object manipulation. The "Symball" application [7] allows users to hit balls at each other, although with limited 3D tracking. On their phone screen players can see a table tennis table and a virtual paddle. They select a real color that they would like their phone to track and as they move the phone relative to this color the paddle moves in the x-y direction on the screen. Thus, while intuitive, the "Symball" interaction uses 2D rather than 3D object tracking.. By visually tracking real objects, the camera phone can be used for 6 DOF input. Rohs' Visual Codes [23] is an example of mobile phone barcode reading combined with 3 DOF tracking. By recognizing and tracking a pattern, the phone movements can be estimated and used as input. The pattern can also be associated with phone functions and act as a menu item [24] . Other similar phone applications are the Spotcode [25] and QR-Code pattern tracking systems [21] . Spotcode is a twodimensional ring like bar code that can be tracked in real time with a phone camera. The Spotcode software performs image processing techniques to extract the identity of the pattern and its angular orientation relative to the phone. Similarly QR-Code is a two-dimensional bar code developed in Japan that can also be recognized by mobile phones, although it does not provide full 6 DOF tracking.. Mobile AR started with backpack configurations such as Feiner's Touring Machine [4] . A similar setup, ARQuake [26] , showed how these same systems could be used for outdoor gaming. Both these systems make use of a HMD. At the same time Rekimoto started to experiment with handheld AR. Transvision [22] consists of a small LCD display and a camera. These are connected by a cable to a computer that performs the augmentation. Two users sit across the table and see shared AR content shown on the displays. They can select objects by ray casting and once selected objects are fixed related to the LCD and can be moved. The ARPAD interface [17] is similar, but it adds a handheld controller to the LCD panel. ARPAD decouples translation and rotation. A selected object is fixed in space relative to the LCD panel and can be moved by moving the panel. Rotation is performed using a trackball input device.. The AR-PDA project [5] was the first to use a camera equipped PDA for AR. The video stream was sent to a server for image analysis and rendering of graphics. Wagner developed the first self-contained PDA AR application [30] by porting ARToolKit [2] to the PocketPC. The Invisible Train [29] uses a stylus for interaction with 3D data on a PDA AR application. The user taps rendered objects on the screen with the stylus to change the position of tracks on the train set. Klein used a similar setup for a game on a tablet PC [13] . The movement of the character is controlled with a stylus. The user can also toss plastic tokens in the game area to create effects.. AR on mobile phones took a similar path beginning with client server solutions. The first to explore selfcontained AR on mobile phones were Moehring and Henrysson. Moehring [16] used 3D markers on which a coordinate system was printed. Henrysson ported ARToolKit to Symbian and created an application that augmented a map with the current tram positions derived from a timetable [11] . A first step to towards interaction with 3D data using an AR enabled mobile phone was [9] were two players sitting face-to-face played tennis using the mobile phones as rackets. The interaction is limited to the collision between the device and a virtual ball being simulated in the marker space between the players.. Henrysson conducted a user study [10] comparing different interaction techniques for translation and rotation of 3D objects using a mobile phone with AR. Similar to the AR-PAD the translation and rotation were decoupled and studied separately. The user selects the object using a button on the keypad. Once selected the object is locked relative to the device. Translation and rotation can be performed by translating and rotating the device respectively. Translation and rotation can also be performed using the keypad. Each axis is mapped to two buttons for decrementing and incrementing the transformation. As an alternative for rotation an Arcball was implemented and controlled by moving the device in a 2D plane. The application presented in this paper builds upon this work and look at how to combine different transformation techniques for general object manipulation and scene assembly.. There are several examples of 3D graphics applications on mobile phones. The vast majority are games that provide joystick type control of vehicles and objects in 3D environments. Larsen [14] describe one of the first 3D applications for the mobile phone with more complex object manipulation. This is a brick-modeling program where the user selects and moves virtual bricks using the arrow keys on the phone. This is a client server setup where the rendering of the bricks is made on the server in addition to collision detection. The rendered sub-images are sent to the mobile phone client on which only wireframe rendering is possible. There is no mentioning of interactive change of the view. Transformation is restricted to 2D translation.. Hachet [6] has developed a 3 DOF bimanual camera based interface for interaction both on the device itself and for using a PDA as a 3D mouse. The approach is similar to ours in that it establishes the position and orientation of the device by analyzing the video stream captured by the camera. For the PDA there is also "3D Blockout" [1] , which is a falling block game similar to Tetris. Since the block is falling there is only need for 5 DOF. The interface consists of a menu bar to the right of the screen. The user can move and rotate the block while it is falling to the floor. Watsen et al. [32] has implemented a 6 DOF interface on a PDA for interaction with CAVE-like Virtual Environments. The user controls each degree of freedom with a one-dimensional slider.. Although our application is based on ARToolKit [2], the real time performance of some of these systems led us to believe that we should also be able to get good performance from our code. The mobile phones have developed so much in recent years that there is no need for client server setups for either basic 3D rendering or the image analysis required for 6 DOF tracking. We believe it is now time to explore the possibilities of scene assembly and 3D object manipulation on mobile phones using the camera tracking as input.. The use of AR is not essential for a 3D application, but it is convenient when using camera tracking of markers. If the marker is lost the graphics disappear and the user can adjust the orientation of the device given the video feedback. As stated before it also gives more reference points when navigating the scene.

B I O O O O O O O O O O B I O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Probabilistic analysis of interconnect coupling noise Abstract-Noise simulators and noise avoidance tools are playing an increasingly critical role in the design of deep submicron circuits. However, noise estimates produced by these simulators are often very pessimistic. For large, high-performance industrial ICs, which can contain hundreds of thousands of nets, the worst case estimates of the noise results in thousands of reported violations, without any information about the likelihood of the possible noise violation. In this paper, we present a probabilistic approach to prioritize the violating nets based on the likelihood of occurrence of the reported noise. We derive an upper bound on the probability that the total noise injected on a given victim net by a specific set of aggressors exceeds a threshold. This is equivalent to a lower bound on the expected number of clock cycles required to realize the noise violation for the first time, i.e., mean time-to-failure. If the probability of a failure in a victim is sufficiently small, it is possible that even during the operation of the part for a number of years, the probability of failure on the net is negligible and the net can be assigned a lower priority for the application of noise avoidance strategies. We demonstrate the utility of this approach through experiments carried out on an large industrial processor design using a state-of-the-art industrial noise analysis tool. N OISE DUE to capacitive coupling of interconnects has become an important reliability issue in the design of deep submicron (DSM) circuits. Advances in process technology have reduced the wire widths and wire spacing, resulting in taller, thinner, and closer wires [1] . The crosscoupling capacitance between wires has become a dominant component of the total wire capacitance [1] - [4] , and can cause a degradation in performance or a malfunction of the circuit. A glitch can occur on an otherwise stable net (victim) due to switching of its neighboring nets (aggressors) and can then propagate to a storage element or a dynamic node, which could alter the circuit state. Noise causing this type of failure is referred to as functional noise. Alternatively, if the victim net is also switching, then the noise injected onto the victim by the aggressors can result in a significant change in the delay [2] , thereby impacting the clock frequency of the circuit. This type of noise is referred to as delay noise. For static circuits, functional noise may not be as severe as delay noise. However, the increasing use of dynamic circuits makes consideration of functional noise equally important. In addition, the likelihood of either type of malfunction has increased significantly due to lower supply voltages and higher clock frequencies. Thus, methods for accurate estimation of functional and delay noise and techniques for reducing their effects have become critical in the design of DSM circuits. Key features of noise simulators [5] , [6] include: 1) models of the network consisting of the victim and its aggressors, as well as models of the gates that drive them; 2) methods for estimating the resistance and capacitance of each net and the coupling capacitance to neighboring nets; 3) criteria for filtering out insignificant aggressors of each net; 4) methods for computing the noise threshold at the input of the victim's receiver gate; and 5) a simulation engine that numerically solves the network equations to compute the voltage waveform of the noise induced on the victim.. As with any aspect of circuit design, there is a tradeoff between the accuracy of the noise estimation techniques and the time required to compute them. Fast techniques are essential since many high-performance industrial circuits can have hundreds of thousands of nets. Consequently, most often each cluster consisting of a victim and its associated set of aggressors, is modeled as a linear network, so that the composite noise waveform due to all of the aggressors can be computed as the sum of the noise waveforms due to each aggressor.. Even with linear models of the nonlinear drivers [6] , [7] , the time required for SPICE-level simulation of a fully distributed RC network is unacceptably high, and hence, reduced order models [8] , [9] are used in noise simulators. For constructing noise avoidance strategies or for delay analysis in the presence of interconnect coupling, further simplifications are necessary. For example, the coupling capacitance ( ) can be replaced with an equivalent grounded capacitance in the range of in static timing analysis [10] . Closed-form expressions for the noise waveform have been developed under a variety of simplifying assumptions, and have been applied to noise avoidance using driver sizing, wire spacing, and wire sizing [11] - [16] .. Regardless of the models and the approximations used, noise analysis tools are designed to be pessimistic due to the consequences of missing a potential malfunction. However, in large industrial designs, this has resulted in thousands of reported failures, which are expensive to fix in terms of chip area, chip performance, and design time. The noise injected onto a victim by an aggressor during a clock cycle is a time-varying function that depends on the pair of input vectors applied at cycles 0278-0070/03$17.00 © 2003 IEEE and , the transition times of the signals and the time at which signal transitions occur. In the case of delay noise, the switching and transition times of the aggressor depend not only on its gate and wiring delays, but also on the waveform at the victim's output [17] . The root cause of the pessimistic estimates of noise is the way in which the noise waveforms from each aggressor net are combined to form the composite noise waveform. A common approach [5] - [7] is to assume that all aggressor nets can switch in the same direction and at the same time, and that their peaks will be aligned. This results in a worst case maximum height composite noise waveform. Often, however, the number of aggressors with significant amount of coupled noise can be large, exceeding five to ten aggressors. In such a case, the likelihood of realizing a worst case composite noise, where all aggressors are required to switch at exactly the right time, is small.. A number of deterministic methods have been proposed to more accurately estimate the noise waveform by accounting for the logic [18] - [20] and timing relationships [5] , [21] - [24] between the aggressors and a victim. One approach is to perform static timing analysis (ignoring the functional information) and construct timing windows which specify the earliest and latest time that a signal can switch within a clock cycle. The peaks of the two aggressors would be aligned only if their timing windows overlap [6] . For delay noise, determining the alignment of the aggressors and relating that to the maximum delay is more involved. In [22] , an convergent iterative algorithm is presented that accounts for the fact that the aggressor timing window depends on the victim's output.. Noise estimates can be improved as well by taking into account logic correlations [6] , [23] - [26] . For example, when considering functional noise, even if the timing windows of two aggressors overlap, can be determined that the two signals never switch in the same direction, then the noise value is taken as the maximum of the respective peak values instead of the sum of their peak values. In [26] , a precise characterization of such signal interactions based entirely on the logic or functional characteristics is given. This allows identification of signals that can interfere as input to noise avoidance tools. In [23] , the problem of identifying the vector pair that would result in two nets switching within a given interval is formulated as a constrained optimization of a pseudo-Boolean function.. Although the incorporation of timing information in noise analysis improves the accuracy of the noise estimates, typically, the timing windows obtained from static timing analysis are wide and eliminate only a small portion of the reported failures. In addition, for large circuits, only a limited amount of logic correlations can be derived because physical proximity of nets and logic dependencies between them need not be related. Moreover, these methods require high computational effort. Even after accounting for logic and timing correlations, noise reports for high-performance circuits with hundreds of thousands of nets often identify thousands of nets with potential violations, and provide only the worst case noise associated with each victim. Deterministic methods, therefore, can not completely address the problem inherent in the pessimistic assumptions of coupled noise analysis.. In this paper, a method for estimating the likelihood that a total noise (functional) on a victim net will exceed a given threshold is presented. If the noise on a victim net originates from a significant number of aggressors, the likelihood of all aggressors combining in a worst case manner is small. If this likelihood is sufficiently small, it is possible that even during operation of the part for a number of years, the probability of a failure (i.e., a noise violation) on the net is negligible, and the net can be assigned a lower priority for the application of noise-avoidance strategies or even eliminated from further consideration. The probability of a failure on a victim net will decrease as the number of aggressor nets involved increases and as their timing windows become wider. One simple criterion to prioritize nets is the expected number of clock cycles before the first violation occurs on a specific victim. We refer to this quantity as the net's mean time-to-failure (MTF).. It is important to note that there is a basic distinction between the approach taken here and the classical statistical approach that might be used to predict noise. In the latter, the set of all nets is viewed as an ensemble and noise associated with a net is viewed as a random variable over the ensemble. Such an approach would not lead to accurate statements about any specific net. The approach described here characterizes a victim by its set of aggressors, each of which contributes a specific noise waveform (a deterministic function of time obtained from simulation) to the victim. The randomness or variability of the noise on a victim net is over different clock cycles and arises due to the times when each aggressor switches over a specified time interval within a clock period.. The rest of the paper is organized as follows. Section II contains a description of the models. A discussion of the assumptions appears in Section III. Sections IV and V contain the main technical contribution, namely, the upper bounds on the probability that the noise on a victim will exceed a given threshold, and estimates of its MTF. Results of experiments carried out on a large, high-performance PowerPC microprocessor are presented in Section VI. A brief description of a recently developed industrial noise simulator called ClariNet [6] , [7] , which was used in the analysis is also given in this section. Section VII contains a summary.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


TIGHT BOUNDS FOR THE FIRST ORDER MARCUM Q-FUNCTION In this paper, we develop new bounds for the first order Marcum Q-function, which are extremely tight and tighter than any of the existing bounds to the best of our knowledge. The key idea of our approach is to derive refined approximations for the 0 th order modified Bessel function in the integration region of the Marcum Q-function. The new bounds are very tight and can serve as an effective means in bit error rate (BER) performance analysis for non-coherent demodulation in digital communication.Keywords: BER, Marcum Q-function, upper bounds, lower bounds, the 0 th order modified Bessel function of the first kind.

O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O B B I B I B I O O O O O O O O O O O