Adaptive kernel approach to the time series prediction Abstract This short article describes two kernel algorithms of the regression function estimation. One of them is called HASKE and has its own heuristic of the h parameter evaluation. The second is a hybrid algorithm that connects the SVM and HASKE in such a way that the definition of the local neighborhood is based on the definition of the h-neighborhood from HASKE. Both of them are used as predictors for time series. Estimation of the regression function is one of the basic problems that deals with the discipline called machine learning [22, 26] . The aim of the evaluation of regression function is to find some dependencies between variables in the observed dataset. Sometimes these relations can be overt like the dependence between the current intensity and the voltage in the linear element (Ohm's law), but in many cases this dependence is hidden or difficult to notice.. There are two main groups of methods of the regression function estimation: parametrical and non-parametrical methods. Parametrical methods can be described as models with a well-defined functional form with finite number of free parameters, and which values must be established. Usually some optimization criterion is defined and values that optimize it are admitted as ''proper''. The parametrical model can also be written in the following way:. where e y is the estimator of the variable y, x is the independent variable and h is the vector of the model parameters.. Methods from the second group are also described by some free parameters, but the equation of the value estimator should not be concerned as the description of the variable's behavior. Non-parametric estimators do not make any assumptions about the functional form of dependencies between independent and dependent variables. In other words, non-parametric estimator approximates values, but does not try to explain the nature of the dependence. This leads to the general form of the nonparametric estimator:. where the function f is unknown and is the object of the research. Very common non-parametric regression function estimators are spline functions [4, 9] , radial basis functions [13] , additive (and generalized additive) models [12] , the LOWESS algorithm [3] or kernel estimators [17, 28] with support vector machines [1] .. The specific kind of data-time series-can be analyzed by the usage of typical methods like autoregressive models, decomposition method or the Fourier analysis. As it will be shown in this article, kernel methods can also be useful as a time series prediction tool, although some standard parts of algorithms should be changed (the method of the smoothing parameter h evaluation [16] .. In this article, author describes two new kernel methods of the time series prediction and both of them are based on the certain regression function estimation. The first method-a kernel estimator HASKE-is described based on the Nadaraya-Watson kernel estimator, but can also be based on other estimators from this group. This estimator applies the new adaptive method of smoothing parameter evaluation, using the definition of the h-neighborhood. This new method avoids the big estimation error in a case, where there are no training objects in the neighborhood of the test object. The algorithm is specially designed for time series with the visible periodic dependence between past and present values that is often determined by the nature of the time series. The results of HASKE algorithm are compared with other kernel estimators and the well-known decomposition method.. The second kernel predictor-called the HKSVR-is a hybrid algorithm that connects the mentioned group of kernel estimators and the support vector machine. It combines the adaptive definition of the test point neighborhood from HASKE with advantages of the support vector machine regression, but should not be considered as the extension of HASKE. The HKSVR is especially designed for the time series for which we cannot point the easy interpretable dependence between past and present values. As the HKSVR can be classified as the local support vector regression, its results are compared with that of the support vector machine.. Both algorithms were widely presented in the CORES conference [16] .. 2 Non-parametric estimators of the regression function The article describes two new algorithms of the time series prediction. Both of them belong to the group of non-parametric methods and base on new definitions of neighborhood and locality. The prediction problem is brought to the estimation of the regression function in the modified data space. The first algorithm (HASKE) defines the notion of h-neighborhood and helps to avoid the effect of its emptiness. Two required parameters of this algorithm are calculated adaptively with the usage of train and tune sets. The results of HASKE were compared with the results of the kernel regressor and the decomposition method. This algorithm is designed for time series that have well-defined time dependency (the value of the period is easy to interpret) like it is in the case of the G series (amount of passenger has the 12-month period). On the basis of this value the transformation to the new space is performed. Moreover, the data in the modified space should have significant correlation. If this condition is not fulfilled, the result of HASKE may not be satisfactory. It can be observed in the case of the results for E series prediction.. Second model (the HKSVR) is the local hybrid connection of the SVR and HASKE. It was tested on the real financial data. Generally, this algorithm is correct for time series that do not have the dominating period. That is the reason, why the other methods are used to find the value that defines the transformation of time series to the new space, for example the Fourier transformation. The algorithm improves the prediction for the short time horizons, excluding the next value prediction. Applicability of this model depends on the data correlation in the modified space. The normalized criterion Q [ (0, 1) describing the improvement of prediction was used as a comparison tool. Values higher than 0.5 indicate that the usage of the HKSVR was appropriate.. Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited. Performing time series prediction as the kernel estimation of the regression function may meet the problem of empty h-neighborhood for test objects. It occurs that typical algorithms of smoothing parameter evaluation fail in respect to time series prediction. Results of experimentssome of them are shown in Table 2 -suggest to modify the value of smoothing parameter in such a way that for every test object its h-neighborhood would be non-empty. Heuristic Adaptive Smoothing Parameter Kernel Estimator Algorithm (HASKE) solves the mentioned problem. The solution is the set of two parameters l and a. Each of them depend on the given time series, but only the first of them is connected with the problem of the empty h-neighborhood. For the given time series its training part is divided into two separate subsets: train and tune. Then the Nadaraya-Watson kernel estimator is trained with respect to the value of the l, and the error of the tune set prediction is observed. The value that gives the lowest prediction error on the tune set is chosen as the optimal value of the l parameter.. HASKE, like other mentioned kernel estimators, is not defined for typical time series space (pairs of observation and time stamp: (t, x t )), but requires the transformation to the new space with clearly defined dependent and independent variables. The typical form of transformation is definition of the time interval Dt, that implies the new set of pairs of observation ðx t ; x tþDt Þ. This assumption requires setting the Dt value, that is usually prediction horizon or the strongest time series period length. HASKE should not be applied when Dt is hard to determine or when the correlation between dependent and independent variables is low.. The support vector regression model described in Sect. 2 is a global one. There is also a number of its modifications that use a local learning paradigm. The algorithm presented in [8] uses kNN as a local training set. The other algorithm says that the value of the parameter depends on local covariance matrix (R i ), calculated on the basis of training points from the neighborhood of point x i [14] .. The heuristic of the smoothing parameter evaluation, presented with the HASKE algorithm, gives more appropriate definition of the test point neighborhood because it makes it possible to reduce the time series prediction error. It is worth to check, whether that new definition of neighborhood improves the results of the support vector regression for time series.. The Hybrid, Kernel and Support Vector Regression algorithm (HKSVR) combines the kernel regression (considered as Nadaraya-Watson estimator or similar) and SVM regression. Initial step of the algorithm determines the neighborhood of train objects for each test point. Second step performs support vector regression for the test point on the basis of its train neighbors.. Similarly, as HASKE, the HKSVR performs prediction as the estimation of the regression function in the modified space. That means that the value of the parameter that defines the transformation is necessary. As distinguished from HASKE, the HKSVR is designed for prediction of time series, where it is hard to point their period length. The length of the time series period may be determined with the usage of the Fourier transformation.. It is important to notice that the HKSVR usefulness depends on the correlation of data in modified space. Usage of the HKSVR brings the prediction improvement if the correlation is significant (close to one).

O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A comparative study of the performance of high resolution advection schemes in the context of data assimilation High resolution advection schemes have been developed and studied to model propagation of ows involving sharp fronts and shocks. So far the impact of these schemes in the framework of inverse problem solution has been studied only in the context of linear models. A detailed study of the impact of various slope limiters and the piecewise parabolic method (PPM) on data assimilation is the subject of this work, using the nonlinear viscous Burgers equation in 1-D. Also provided are results obtained in 2-D using a global shallow water equations model. The results obtained in this work may point out to suitability of these advection schemes for data assimilation in more complex higher dimensional models. Spatial discretization methods for solving partial di erential equations (PDEs) can be broadly classiÿed as ÿnite di erence (FD) [1, 2] , ÿnite volume (FV) [3] , ÿnite element (FE) [4] [5] [6] and spectral methods [7] (including the discontinuous Galerkin (DG) methods [8] ). All of these methods combined with explicit or implicit time integration schemes can be e ectively applied to solve PDEs (of various types such as hyperbolic, parabolic and elliptic).. For numerical solutions of conservation laws, such as the Euler equations in gas dynamics [9] which describe evolution and propagation of ows involving sharp fronts and shocks, several methods have been suggested in the FD, FV, FE, spectral and DG methods literature. Some of the most popular methods in the FV context are Lax-Wendro , Lax-Friedrichs, 721 models (therefore, these are also referred to as forward models). Hence a good forecast requires that these NWP models represent accurately the dynamics of the atmosphere (including the oceans) and the initial conditions supplied for integration (of the NWP models) be known accurately. Estimation of the state or evolution of the atmosphere using the information provided by NWP model prediction and observations of the atmosphere is carried out by data assimilation (DA) [30] . Current DA methods are based on either using results from estimation theory (such as application of Kalman, extended Kalman, ensemble Kalman ÿltering, for details, once again, please see Reference [30] and references therein) or variational methods, which are based on minimization of a cost functional which measures the distance (in a suitable norm) between observations and NWP model forecasts. The objective of variational data assimilation is to determine a model trajectory (by adjusting initial conditions used for model integration) that satisÿes the model equations as a (strong or weak) constraint while simultaneously minimizing the lack of ÿt between model predictions and heterogeneous observations in a least-squares sense. Please see References [32, 33] for further details. Large scale minimization algorithms which require availability of gradient of the cost functional with respect to the control variables (provided e ciently by adjoint methods, which are integrated backwards in time) are used for this purpose.. The impact of di erent discretization techniques for the advection term(s) in the framework of inverse problems and problems related to DA have not been extensively tested, except for work by VukiÃ ceviÃ c et al. [24] and Thuburn and Haine [34] . In Reference [24] the authors performed DA experiments to reveal the relationships between their properties with respect to data assimilation with three di erent (central di erence: LEAPFROG, MPDATA, QUICK) schemes for the advection of a passive tracer in two dimensions using a linear 2-D transport equation. Their results indicate that more accurate advection schemes need to be used to solve both, forward and adjoint models in time to achieve higher accuracy regarding recovery of initial conditions for data assimilation; also the same discretization scheme should be applied consistently both for forward and adjoint model integrations. Thuburn and Haine [34] recall Godunov' s theorem (which states that any linear monotonic advection scheme cannot provide more than ÿrst-order accuracy), they studied the a ects on adjoint sensitivity computations using a nonlinear, nonoscillatory (QUICK) scheme on a one-dimensional linear advection equation model. They also suggest modiÿcations to advection schemes to obtain adjoint sensitivity results that are meaningful (in the particular physical setting considered by them). In this context, a total variation diminishing (TVD) scheme based on a slope limiter has been suggested.. Since the mathematical models used to study uid ows and weather prediction are highly nonlinear, as a step towards understanding the e ects of using high order advection schemes in DA, we study in this work the impact of using FV methods that are slope limited using van Leer type and PPM for spatial discretization (in 1-D and 2-D). In one dimension, a nonlinear viscous Burgers equation model and in 2-D the spherical global shallow water equations have been used as proxy for more complex NWP models. We show that for a particular smooth initial condition, we obtain a smooth solution for these model problems (in the context of smoothness property of geophysical ows as discussed above), and implement the adjoint method to conduct DA experiments.. The paper is organized as follows. In Section 2 we present the forward model as well as describe numerical solution of the nonlinear Burgers equation using FV discretization. Section 3 describes the test case considered along with results obtained using several slope limited TVD schemes as described in Section 2. Section 4 describes the derivation of the adjoint and tangent linear models (which are used for DA) and veriÿcation of these discrete models. Section 5 provides a brief description of the minimization algorithm used. The performance of the various slope limited and PPM schemes for the minimization of a certain cost functional, in other words, in DA experiments (1-D and 2-D) is presented in Section 6. Finally, in the section of summary and conclusions we discuss the impact of the di erent advection schemes in the framework of our numerical results. We have studied the impact of various high resolution TVD, FV (which use MUSCL slope limiters and PPM) schemes on data assimilation for two nonlinear model problems, namely will focus on further investigation of validity of the above ÿndings for a higher dimensional system with real observations. Another aspect of interest we aim to address is to compare how the error covariance matrices are propagated by these various limiters in the context of sequential variational DA, in particular, in an ensemble Kalman ÿltering framework. Of interest to the computational uid dynamics community is the issue of uid ow control, where the numerical model used for solving the governing equations are solved by FV methods, using various slope limiters and, or the PPM scheme. It should be noted that in order to compute the adjoint variables in the backward direction, we require forward states to be available (as evident from the above piece of adjoint code) in memory or recompute them, see research on checkpointing [76, 77] for discussion on the trade-o between storing in memory and recomputation.

O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


DECODE: Exploiting Shadow Fading to DEtect COMoving Wireless DEvices Abstract-We present the DECODE technique to determine whether a set of transmitters are comoving, i.e., moving together in close proximity. Comovement information can find use in applications ranging from inventory tracking to social network sensing and to optimizing mobile device localization. The positioning errors from indoor RSS-based localization systems tend to be too large, making it difficult to detect whether two devices are moving together based on the interdevice distances. DECODE achieves accurate comovement detection by exploiting the correlations in positioning errors over time. DECODE can not only be implemented in the position space but also in the signal space where a correlation in shadow fading due to objects blocking the path between the transmitter and receiver exists. This technique requires no change in or cooperation from the tracked devices other than sporadic transmission of packets. Using experiments from an office environment, we show that DECODE can achieve near-perfect comovement detection at walking speed mobility using correlation coefficients computed over approximately 60-second time intervals. We further show that DECODE is generic and could accomplish detection for mixed mobile transmitters of different technologies (IEEE 802.11b/g and IEEE 802.15.4), and our results are not very sensitive to the frequency at which transmitters communicate. M ANY location-aware applications benefit from higherlevel information about the movements of transmitters. One instance of such higher-level information is comovement, which describes whether a set of transmitters is moving together on a common path. Comovement information could be used to infer containment relationships, indicating, for example, that two devices are owned and carried by the same person, or that several tagged objects are placed on the same pallet. It could also be used to infer social relationships if the transmitters are carried by different persons or for optimizing localization system performance.. While it is straightforward to derive comovement relationship from position coordinates and trajectories generated by a localization system, sufficiently accurate and precise data are not always available. Indeed, our evaluation of a Bayesian WiFi localization system (M1) [1] shows that the location estimation errors lead to bias and variance in the euclidean distance between two comoving transmitters, making detection of comovement difficult. Global Positioning System (GPS) accuracy is also frequently degraded in urban canyons [2] , and even if signals are available, GPS receivers are not commonly used in portable devices due to their high energy consumption. For indoor environments, localization systems require the presence of multiple landmarks or receivers, which adds infrastructure cost. Coarse comovement information can also be obtained from connectivity through short-range radios [3] . This, however, requires tracking software to be installed on all mobile devices; it cannot easily be inferred through infrastructure solutions alone. In this work, we presented DECODE, a system that detects comoving wireless devices. DECODE's strategy is founded on exploiting the similarity in shadow fading for the packets transmitted from a set of transmitters. We showed that our technique can work in both the signal space and its corresponding location space, but that the signal space approach provides the key advantage that only a single base station is needed. Our approach was general in that it could detect comovement of wireless transmitters with different radios. We validated our approach through simulations for various indoor and outdoor environments. We further demonstrated that it works for both IEEE 802.11b/g WLAN and IEEE 802.15.4 Mote devices in real indoor environments.. Given 130 seconds of mobile data, DECODE can achieve a true positive rate of 100 percent with 0 percent false positive estimated over 1,071 data subsets. However, a key finding of this work is that mobility is critical for our approach, and that the DECODE's effectiveness scales with both the time and speed of the devices mobility. We also showed that DECODE's performance is insensitive to the sampling rate and a sampling rate of 1 packet/second for 130 seconds was sufficient to achieve a 100 percent detection rate and 0 percent false positive rate.. Finally, because DECODE's effectiveness is quite sensitive to mobility, we examined using the RSSI variance for mobility detection. We found this technique to have a mobility detection rate of over 96 percent with the corresponding false positives to be less than 1 percent. Therefore, detecting mobility has a straightforward solution and does not limit the DECODE system. In this section, we discuss the feasibility of detecting transmitters that are static and located within close proximity. We continue the discussion by giving out the advantages of operating in the signal space in comparison to the location space. We finally conclude this section by discussing the impact of missing samples on comobility detection. The previous works on detecting colocated and comoving objects have either been based on absolute location of the transmitters obtained using localization indoors and GPS outdoors or from proximity sensing using short range infrared (IR) or Bluetooth devices. We know of no other work that infers colocation or comovement directly from signal strength measurements. In this section, we classify the related work into three main categories.. Mobility detection. Several earlier studies have concentrated on distinguishing mobile and stationary transmitters. Sohn et al. [8] determine mobility from GSM traces using seven different metrics one of which is the variance in Signal Strength which is similar to our approach. Similarly, Muthukrishnan et al. [9] discuss detecting mobility from RSSI in WLAN. LOCADIO [10] again used variance to detect mobility and combined it with a two state Hidden Markov Model (HMM) to eliminate oscillations between the static and mobile states. We build on this work-detecting mobility is an integral component of the DECODE technique.. Proximity-based inference. Proximity based colocation inference techniques mainly consist of using short-range IR or Bluetooth devices to estimate distance between the transmitters. The Reality Mining project [3] [11] used Bluetooth-capable GSM phones to record the other nearby Bluetooth devices and transmit them to the central server for inferring social interaction patterns. SpotOn system [12] used radio signal attenuation to estimate the relative distance between the special tags. Though these techniques look attractive for colocation detection, they require tracking software on the devices themselves and are effective only for detecting devices that have the same technology. Our scheme is more generic as it involves measurement of RSSI which is common to GSM, WLAN, Zigbee, and Bluetooth.. Distance threshold detection. This baseline detection technique involves estimating the locations of different transmitters and deriving conclusions about comovement based on the distance between the estimated positions of the transmitters. Recent efforts have resulted in a plethora of methods to determine the locations of transmitters. Want et al. [13] used infrared and [14] employed ultrasound to perform localization. However, both of them required specialized infrastructure to be deployed for performing localization. On the other hand, in spite of meter-level accuracy [15] , using RSS [1] , [16] , [17] is an attractive approach because it can reuse the existing wireless infrastructure.. RADAR [16] , the first algorithm for IEEE 802.11 transmitters in this category, uses RF Fingerprint information (vector containing known locations of transmitter along with a measure of the observed signal strength at different receivers) observed at three receivers and performs a nearest neighbor matching algorithm to determine the location of the transmitters with a three meters median accuracy. Elnahrawy et al. [1] use Bayesian learning algorithm on RF fingerprints observed at three or more receivers to obtain a median 802.11 localization accuracy of 3-4 meters. The most accurate 802.11 location system to date is [18] which uses the Hidden Markov Model and Bayesian inference derived from observations at nine different receivers yielding a median accuracy of 1 meter. Further, the average localization accuracy employing RSS in a 802.15.4 (Zigbee) network [17] and an active RFID system [19] is about the same with median errors around 3-4 m when using four receivers.. While the recent papers [20] , [21] have reported a higher accuracy localization techniques, these techniques require transmitters to perform synchronized communications which is not common across typical transmitters that we analyze in this paper. Further, these papers have not reported the accuracy in a mobile environment questioning its applicability for the detection of comovement.. Intuitively, one can derive comovement information with threshold detection on the distance between two transmitters. Compared to DECODE (in signal space), all these localization systems require three or more receivers to work in concert, whereas DECODE can be used even with only one receiver. In addition, the accuracy results reported for WiFi localization raise questions about the precision of such a detection approach. We will further address this question in the next section.

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O


VSim: Simulating multi-server setups at near native hardware speed Simulating contemporary computer systems is a challenging endeavor, especially when it comes to simulating high-end setups involving multiple servers. The simulation environment needs to run complete software stacks, including operating systems, middleware, and application software, and it needs to simulate network and disk activity next to CPU performance. In addition, it needs the ability to scale out to a large number of server nodes while attaining good accuracy and reasonable simulation speeds. This paper presents VSim, a novel simulation methodology for multi-server systems. VSim leverages virtualization technology for simulating a target system on a host system. VSim controls CPU, network and disk performance on the host, and it gives the illusion to the software stack to run on a target system through time dilation. VSim can simulate multiple targets per host, and it employs a distributed simulation scheme across multiple hosts for simulations at scale. Our experimental results demonstrate VSim's accuracy: typical errors are below 6% for CPU, disk, and network performance. Real-life workloads involving the Lucene search engine and the Olio Web 2.0 benchmark illustrate VSim's utility and accuracy (average error of 3.2%). Our current setup can simulate up to five target servers per host, and we provide a Hadoop workload case study in which we simulate 25 servers. These simulation results are obtained at a simulation slowdown of one order of magnitude compared to native hardware execution. Simulation is a vital tool in contemporary experimental research and development, both in hardware and software design. Because simulation is so widely used for different purposes, there exists a whole range of simulation techniques. Emulation or functional simulation is at one end of the spectrum and models the functional aspects of a computer system only. The simplest form of emulation is interpretation in which the simulator interprets one instruction at a time [Austin et al. 2002] . Dynamic model 1Gbit and 100Mbit target networks within 4.9%; and we model SSD (Solid State Drive) and HDD (Hard Disk Drive) disks within 3.3% and 5.5%, respectively. When put together, VSim can simulate complete computer systems running complete software stacks with good accuracy. We demonstrate this through a client-server setup running the Lucene search engine which involves simulating a client and a server running an operating system, Java virtual machine, network protocol stack and disk I/O (average error of 3.2%). We also consider case studies involving more complex setups, including the Olio Web 2.0 benchmark running a web server, file server and database server, as well as a 25-server Hadoop workload setup. VSim achieves good accuracy while incurring a simulation slowdown of one order of magnitude only compared to native hardware execution. Moreover, VSim can simulate multiple target servers per host server (up to five target servers per host server in our current setup with an average error below 5%).. Given that VSim can simulate large systems on smaller systems with good accuracy at good speed while running complete software stacks, we believe that VSim is a promising approach towards simulating systems at scale. Moreover, VSim is complementary to other simulation paradigms, i.e., whereas cycle-accurate simulation remains an important tool in the computer designer's toolbox for making detailed microarchitecture design decisions, VSim is a more appropriate tool for making system-level design decisions in multi-server setups. Finally, both software developers and system architects can benefit from VSim. Software developers can use VSim to evaluate software optimizations and tune their software before deployment on a target system. System architects can use VSim to evaluate server design trade-offs and make system-level design decisions. Service providers can potentially use VSim to find performance bottlenecks in existing systems: they can deploy an existing software stack on a VSim simulation model and then vary both hardware and software parameters to identify performance issues. This paper is organized as follows. Section 2 presents and describes the VSim simulation paradigm in detail. Section 3 then elaborates on the experimental setup which we use to evaluate VSim in Section 4. Finally, we describe related work (Section 5), and we conclude (Section 6). Simulating multi-server systems is challenging. It requires the ability of running complete software stacks, while modeling CPU, disk, and network activity. In addition, it needs good simulation speed and accuracy, while being scalable. This paper presented VSim, a novel simulation paradigm which offers a unique trade-off in speed versus accuracy versus scalability while being able to model complete systems, a promising approach for simulating systems at scale. VSim leverages virtualization technology to run multiple target servers as guests on a host and manipulate CPU, disk, and network performance as observed by the guests through time dilation such that the software stacks are given the illusion of running on the target system. The implementation of VSim in VirtualBox and the evaluation presented in this paper illustrate its accuracy: 2.0%, 4.4%, and 4.9% average error for modeling CPU, disk, and network performance, respectively; complete workloads (Lucene and Olio) involving CPU, disk, and network activity are shown to be accurately modeled in VSim (average error of 3.2%). These results are obtained at a simulation slowdown of only one order of magnitude compared to native hardware speed, and our current implementation can simulate up to five target servers per host. We reported on two case studies illustrating how VSim can be used for making design trade-offs and for exploring workload scaling behavior. The simulation approach most closely related to VSim probably is COTSon. COTSon [Argollo et al. 2009 ] is an open-source simulation framework developed by HP Labs. COTSon targets cluster-level systems consisting of multiple multicore processor nodes connected through a network, i.e., it targets both scale-up (i.e., multicore and manycore processor simulation) as well as scale-out (i.e., simulation of a multinode cluster). COTSon uses the AMD SimNow full-system simulator to functionally simulate each node in the cluster. Each COTSon node further consists of timing models for the disks, network card interface and the CPU (i.e., processor and memory). The various COTSon nodes are interconnected through a network mediator. There is at least one key difference between VSim and COTSon. VSim models target performance by moderating the amount of work done per simulation quantum for each of the simulated targets. COTSon, on the other hand, uses a performance model that is fed by a dynamic trace of instructions which is analyzed and based on which performance is estimated. These differences lead to different trade-offs in the simulation space. VSim is faster (10× slowdown versus 50× for COTSon) and consumes much less memory (20MB memory overhead for VSim versus 2GB for COTSon), hence, VSim can simulate both more targets per host and larger systems at higher speed compared to COTSon-VSim is more scalable. A limitation of VSim is that it is tied to the host server's instruction-set architecture (ISA); this is not necessarily the case for the COTSon approach which leverages functional simulation rather than virtualization.. SimOS [Rosenblum et al. 1997 ] was the first simulator achieving the high execution speeds required to simulate a complete system including the operating system. Binary translation was employed to achieve this high execution speed. SimOS includes CPU, disk an network simulation making it suitable for simulating networked systems. SimOS formed the foundation for the VMWare hypervisor; hardware vendors included special instructions to speed up system virtualization. VSim uses hardware assistance in combination with time dilation to achieve high simulation speeds, and it enables simulating multiple guests on a single host.. Open Cirrus [Avetisyan et al. 2010 ] is an open cloud-computing research testbed that was initiated by a collaborative group of researchers in both industry and academia. Open Cirrus' primary goal is to provide a distributed set of federated datacenters as a testbed for system-level cloud computing research: It provides open-source software stacks and APIs, it enables systems-level research; it provides experimental data sets and it allows for studying application development for cloud computing. Gupta et al. [2006] propose time dilation for network simulation. Time dilation provides the illusion to an operating system and its applications that time is passing at a rate different than physical time. They implement time dilation in the Xen virtual machine by filtering delivered timer interrupts. Gupta et al. use time dilation for simulating network devices only. VSim in contrast models complete computer systems, including CPU, network and disk activity through time dilation. Moreover, VSim enables running multiple target servers per host server (this was not explored in the Gupta et al. paper) which is required for simulating scale-out scenarios.. An alternative approach to building simulators and testbeds is to build high abstraction models. For example, Ranganathan and Leech [2007] use utilization traces from real deployments in conjunction with high-level models that correlate resource utilization to power and performance. Meisner and Wenisch [2010] model datacenter workload behavior using queuing models.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O


Community-based ranking of the social web The rise of social interactions on the Web requires developing new methods of information organization and discovery. To that end, we propose a generative community-based probabilistic tagging model that can automatically uncover communities of users and their associated tags. We experimentally validate the quality of the discovered communities over the social bookmarking system Delicious. In comparison to an alternative generative model (Latent Dirichlet Allocation (LDA), we find that the proposed communitybased model improves the empirical likelihood of held-out test data and discovers more coherent interest-based communities. Based on the community-based probabilistic tagging model, we develop a novel community-based ranking model for effective communitybased exploration of socially-tagged Web resources. We compare community-based ranking to three state-of-the-art retrieval models: (i) BM25; (ii) Cluster-based retrieval using K-means clustering; and (iii) LDA-based retrieval. We find that the proposed ranking model results in a significant improvement over these alternatives (from 7% to 22%) in the quality of retrieved pages. The past few years have seen the rapid rise of all things "social" on the web -from the growth of online social networks like Facebook, to user-contributed content sites like Flickr and YouTube, to social bookmarking services like Delicious. Whereas traditional approaches to organizing and accessing the Web's massive amount Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. of information have focused on content-based and hyperlink-based approaches (e.g., PageRank [33] , HITS [22] ), these social systems offer rich opportunities for user-based exploration and analysis of the Web by building on the unprecedented access to the interests and perspectives of millions of users.. In this paper, we examine a popular and growing type of userbased social system, namely social bookmarking systems. These bookmarking systems allow users to personally organize web pages, images, videos, and other web media by tagging (or annotating) each resource with simple keywords or phrases. For example, a user could tag the CNN homepage (http://www.cnn.com) with the tags news, politics, and cnn. Individually, each user is now explicitly linked to a Web resource by a tag or group of tags (see Figure 1) . Collectively, the social bookmarking system can aggregate thousands of user's perspectives on each tagged resource to enrich the resource with large-scale socially-generated metadata. The scale of these systems is large; indeed, a recent study by Heymann et al. found that the single bookmarking site Delicious has already led to the tagging of over 150 million web pages [19] . Based on the scale of this fundamentally user-based effort, there has been growing excitement at augmenting traditional content-based and hyperlink-based web search and browsing through the incorporation of tag information from social bookmarking services, e.g., [3] , [6] , [19] , [24] , [41] , [42] , and [43] . Formally, we consider a universe of discourse U consisting of R resources (e.g., images, videos, Web pages, ...), U users, and a vocabulary of T tags. A user u i is linked to a resource r j by the tag t k used to bookmark the resource. This linkage is represented as the tuple (ui, rj, t k ). The notation introduced here and throughout the paper is summarized in Table 1 . Note that users can tag a single resource with multiple tags. Figure 1 illustrates a simple universe composed of two users, two resources, and three tags. In practice, popular social bookmarking services include Delicious for annotating web pages, Flickr for annotating images, and CiteULike for annotating scholarly articles, among many others.. Although these bookmarking services attract a wide range of taggers -who may vary greatly in interest, expertise, and language -we hypothesize that there are implicit groups of users formed around a common community perspective. For example, an image of a Tyrannosaurus rex could be bookmarked by users belonging to the underlying scientist community (e.g., with tags like cretaceous and theropod), and by users belonging to the underlying elementary student community (e.g., with tags like t-rex and meat-eater ). These communities are not explicitly declared nor even obvious to members of each community; for example, two elementary students may belong to the same community through their use of common tags or through their tagging of common resources, even without knowledge of each other.. This hypothesis of implicit community structure is motivated by previous studies of social bookmarking systems. For example, Golder and Huberman [15] found a number of clear structural patterns in Delicious, including the stabilization of tags over time, even in the presence of a large and heterogeneous user population. This stabilization (which might be counter-intuitive, especially in contrast to the tightly controlled metadata produced by domain experts) suggests a shared knowledge in bookmarking systems. These results are echoed by Halpin et al. [16] , who found a power-law distribution for Delicious tags applied to web pages -meaning that in the aggregate, distinct users independently described a page using a common tagging vocabulary. Similar results can be found elsewhere, including [9] , [7] , and [37] . Along this line, there have been some recent efforts to identify clusters (topics) of tags and resources, including: [26] , which mined tag-based topics via association rules; [42] , which iteratively determines user interests and resource topics over a bipartite graph where users and resource are nodes and tagging counts are edges; and [35] , which identifies groups of web resources by clustering them via their tags using both K-means and probabilistic clustering. These results motivate our interest in uncovering hidden communities that could help explain these phenomena. Toward the goal of identifying community in social bookmarking systems, we posit the existence of L communities that are implicit in the universe of discourse U, where each community is composed of users and tags that are representative of the community's perspective. Since community membership is not fixed, we model membership as a probability distribution, where each user has some probability of belonging to any community.. [ In practice, communities are hidden from us; all we may observe are the user, resource, tag tuples (u i , r j , t k ) that are the result of these communities and the categories they have selected. The challenge that we address in the next section is how to recover these hidden communities from the observable tagging data. Based on the proposed community discovery algorithm, we continue in the following section with an investigation of how to support communitydriven ranking over web resources. In this paper, we have proposed and evaluated a generative community based probabilistic tagging model in which users belong to implicit groups of interest (e.g., students, sports fans) and probabilistically select tags with which to bookmark resources. Coupled with Gibbs sampling parameter estimation, the community-based model can automatically uncover these communities of implicitly related users and their associated tags. We have seen how the community-based model improves the empirical likelihood of heldout test data and discovers more coherent interest-based communities compared to LDA. We have also developed a novel communitybased ranking model for effective community-based exploration of socially-tagged Web resources. Compared to BM25, Cluster-based retrieval using K-means clustering, LDA-based retrieval, we find that the proposed ranking model results in a significant improvement over these alternatives (from 7% to 22%) in the quality of retrieved pages.. Although our approach suggests an important role for socially contributed data in advancing information discovery, there are a number of limitations to its application and generalization to social tagging systems at large. First, LDA based approaches, in general, including our CTAG model require global knowledge and perform many iterations to uncover latent variables. Hence, using them online is difficult. Second, our CTAG model, as does LDA, assumes a fixed number of latent variables and does not consider the temporal aspect of tagging. Therefore, it cannot capture growth and evolution. Third, our assumption of global user communities does not capture individual user behavior. In addition, the lack of standard corpora for social tagging data makes evaluating and comparing results of different research methods difficult. And, results based on human judges of individually collected corpora need to be verified for generalization to different social tagging systems.. However, some of these limitations can be overcome. A combination of a both on-line and off-line approach can solve the processing requirements of LDA-based models. Also, there are methods for dynamically discovering the number of latent variables (see for example [4] ). As future work, we will incorporate time into the CTAG model to capture community evolution over time. We will also investigate ways to construct individual user models in addition to the global user communities. In this section, we evaluate the quality of the Community-Based Tagging Model (CTAG) model and the community-based ranking approach over the social bookmarking system Delicious. We first investigate the quality of the discovered communities using CTAG by comparing each community's tag distribution to tag distributions derived from the non-user based TagLDA approach. Next, we evaluate the proposed community-based ranking to three stateof-the-art retrieval models: (i) BM25; (ii) Cluster-based retrieval using K-means clustering; and (iii) LDA-based retrieval.. For the dataset, we crawled Delicious starting from a set of popular tags. The crawler has discovered 607,904 unique tags, 266,585 unique Web pages annotated by Delicious, and 1,068,198 unique users. Of the 266,585 total Web pages, we have retrieved the full HTML for 47,852 pages. We filter this set to keep only pages in English with a minimum length of 20 words, leaving us with 27,572 Web pages with 16,216 unique tags. We preprocessed both corpora by removing rare tags and users (with only a single occurrence), and stemming the tags.

O B O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


DIM, a portable, light weight package for information publishing, data transfer and inter-process communication The real-time systems of HEP experiments are presently highly distributed, possibly on heterogeneous CPU's. In many applications, there is an important need to make information available to a large number of other processes in a transparent way. For this purpose the "RPClike" systems are not suitable, since most of them rely on polling from the client and one-toone connections. DIM is a very powerful alternative to those systems. It provides a named space for processes to publish information (Publishers) and a very simple API for processes willing to use this information (Subscribers). It fully handles error recovery at the Publisher and Subscriber level, without additional software in the application. DIM is available on a large variety of platforms and operating systems with C and C++ bindings. It is presently used in several HEP experiments, while it was developed in the DELPHI experiment and is maintained at CERN. We shall present its capabilities and examples of its use in HEP experiments in domains ranging from simple data publishing to event transfer, process control or communication layer for an Experiment Control Package (SMI++). We shall also present prospectives for using it as communications layer for future experiment's control systems. The DIM (Distributed Information Management) [1] system was designed in order to provide a communication layer to all process involved with the different tasks of Delphi's Online System. The Online System is responsible for Data Acquisition, Trigger, Control, Monitoring, User Interfacing, etc. Delphi's Online System is composed of about 500 processes distributed over around 50 different machines running mainly VMS and the OS9 Operating Systems. In order to fulfill their tasks these processes need to communicate efficiently and reliably across the different machines. DIM Provides transparent distributed communications to all the processes involved in the different activities of the Online System.. Even though DIM was developed for Delphi, it was designed in a generic fashion so that it could be used widely in other platforms and for other applications. It is currently being used by other experiments: NA50, L3, the L3 Cosmics sub-experiment and BaBar. DIM provides efficient and reliable inter-process communications across different platforms. Its communication mechanism is based on the publish/subscribe method and allows for asynchronous communications and multiple destination updates.. DIM is responsible for basically all communications inside the DELPHI Online System, in this environment it makes available around 30000 services provided by 450 servers.. DIM is also being used by other experiments at CERN (L3, L3 Cosmics and NA50) and by BaBar at SLAC. Other projects (like the Atlas Data Acquisition) decided not to use it directly but to adopt a similar philosophy [13] . DIM has recently been adopted as an interim solution to be proposed to LHC sub-detectors (for their problems of communications within the control system) by the CERN IT Division's Controls Group.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


JANUS: A Framework for Scalable and Secure Routing in Hybrid Wireless Networks Abstract-Hybrid networks consisting of cellular and Wi-Fi networks were proposed as a high-throughput architecture for cellular services. In such networks, devices equipped with cellular and Wi-Fi network cards access Internet services through the cellular base station. The Wi-Fi interface is used to provide a better service to clients that are far away from the base station, via multihop ad hoc paths. The modified trust model of hybrid networks generates a set of new security challenges as clients rely on intermediate nodes to participate effectively in the resource reservation process and data forwarding. In this paper, we introduce JANUS, a framework for scalable, secure, and efficient routing for hybrid cellular and Wi-Fi networks. JANUS uses a scalable routing algorithm with multiple channel access, for improved network throughput. In addition, it provides protection against selfish nodes through a secure crediting protocol and protection against malicious nodes through secure route establishment and data forwarding mechanisms. We evaluate JANUS experimentally and show that its performance is 85 percent of the optimum algorithm, improving with a factor greater than 50 percent over previous work. We evaluate the security overhead of JANUS against two types of attacks: less aggressive, but sufficient for some applications, selfish attacks and purely malicious attacks. C ELLULAR services have developed in the last few years from simple telephony to more sophisticated applications. Internet access and mobile multimedia services are available to numerous clients using laptops or PDAs, via special network cards, or mobile phones enabled to act as wireless modems. However, the availability of 3G services depends on the transmission power of the base station. The use of cellular services to their full potential is impeded by the sharp decrease of the achievable cellular rate as a device moves away from the base station. In addition, in metropolitan environments, where 3G services are expected to be mostly in demand, signal propagation can be extremely erratic, even for low-bandwidth transmissions. Addressing these problems by upgrading current cellular networks may be unappealing to service providers, because of financial reasons and restrictive regulations.. An alternative to overcome the limitations of the cellular services relying only on the base station was proposed in [1] . In this architecture, the cellular network is augmented with Wi-Fi wireless devices organized into an ad hoc network around the existing cellular one. Nodes that have a satisfactory downlink rate from the base station use other nodes in their proximity as relayers. Such an architecture requires nodes to be equipped with two interfaces: one for cellular communication and one for Wi-Fi communication, such as 802.11a/b/g [2] , [3] , [4] . We refer to such a network as a hybrid wireless network.. Although hybrid wireless networks have not been deployed by the industry yet, major industry cellphone companies such as Nokia or Motorola have released several handsets (e.g., Nokia 6136 and Motorola A910 (Martinique)) that have integrated cellular and ad hoc interfaces. We believe the presence of such devices is an incentive to study the security implications of the deployment of such services. Specifically, while providing opportunities for better service to clients far away from the base station, a hybrid wireless network introduces a significant change in the trust and communication model. In the case of a cellular network, the base station represents a centralized point of trust, performing the authentication of all clients and communicating directly with each client without any intermediaries. In the case of a hybrid network, the trust is no longer centralized, as data flows from the base station to the clients via untrusted intermediate relayers. An intermediate relayer may exhibit selfish behavior by avoiding to cooperate in the process of forwarding data to protect its own resources. A relayer may also exhibit malicious behavior by disrupting the functionality of the network, without looking for an immediate benefit for its actions.. Beyond the security concerns raised by the new trust paradigm, a routing protocol has to be scalable and efficient, with respect to both the throughput it offers and the communication and computational overhead it imposes. Clients that resort to multihop paths to achieve an increased downlink bandwidth from the base station have to derive significant gains from this solution compared to using their cellular link. At the same time, as participating devices have limited resources, clients that act as relayers should not be unduly burdened by the protocol or they will refuse to cooperate. In this paper, we have described the challenges of designing a high-throughput, scalable, and secure routing mechanism for hybrid wireless networks. We have proposed JANUS, a framework that addresses these issues with a fast routing core, and several security components. Our framework is flexible and can be tailored to obtain optimal performance both in small and large networks. The security mechanisms provide protection against malicious attackers and, for environments with less restrictive security requirements, against selfish attackers. Our experimental results demonstrate that JANUS can provide hybrid wireless networks with much improved bandwidth without significant overhead. Hybrid wireless networks have been introduced only recently, and research on hybrid routing algorithms has been limited. In this section, we provide an overview of previous research conducted in several areas related with our work: architectures for hybrid wireless networks, ad hoc and hybrid routing, and security for ad hoc and cellular wireless networks.. Architectures for ad hoc and hybrid wireless networks. The most popular model of wireless networks in the literature is that of the ad hoc architecture [25] , [26] , [22] . The distributed nature of ad hoc networks limits their scope, as maintaining a connected network over a large area is quite difficult. There have been efforts to integrate infrastructure-based network models with ad hoc components, but most of them assume single-interface devices. In [27] , GSM terminals are used to relay information to other terminals to improve coverage. In Opportunity Driven Multiple Access [28] , transmission power is conserved by relaying traffic from a CDMA host to the base station through multiple short hops. In [29] , some channels are reserved for forwarding when the fixed channels become congested. In [30] , a generic wireless network is considered, where hosts contact a mobile base station for access outside their cell, using only one interface. In [31] , a hybrid network using the IEEE 802.11 architecture with both DCF and PCF modes is examined, using only one wireless interface. In [32] , multihop paths are used to decrease the number of base stations by increasing their coverage. The overall capacity increases only when two communicating hosts are in the same cell.. Although double-interface architectures are conceptually similar to their single-interface counterparts, they increase the overall capacity by using short-range high-bandwidth ephemeral channels to relay traffic and a long-range lowbandwidth permanent channel to complete operations like routing and data integrity confirmation or as a last resort in the absence of neighbors. The low-bandwidth channels are not necessarily cellular, but the already existing infrastructure makes them attractive options. This architecture has been examined in [1] . In [33] , traffic is diverted to neighboring cells to increase throughput. The use of dedicated stationary relays increases the cost of their solution and limits its utility. A study of local area hybrid networks is presented in [34] . A comprehensive presentation of a rudimentary hybrid network can be found in [35] .. Routing in hybrid wireless networks. UCAN [1] provides two routing algorithms that allow clients to find hosts that are willing to forward their traffic. The first algorithm greedily propagates the routing request to a single neighbor, based on the best rate advertised by its neighbors. The second algorithm is based on a restricted depth flooding to find the best proxy host. However, since both algorithms ignore the finite capacities of the ad hoc links, they do not find the highest throughput path.. Fujiwara et al. [36] provide a unicast routing algorithm that allows mobile hosts to find an alternate path to the base station, when the cellular connection fails. The focus of the work is on reestablishing connectivity to the base station via short paths, in response to emergency situations where direct connections to the base station may become sparse. This is a different goal from JANUS, which attempts to optimize throughput under normal conditions, where all hosts have a direct connection to the base station via the cellular interface.. Security in wireless networks. Security in hybrid wireless networks, although necessary for actual deployment, has been left largely unexplored in the previous work. UCAN [1] focuses on preventing individual hosts from deleting legitimate hosts or adding nonauthorized hosts to the set of relayers that receive credit for forwarding data. It provides protection against isolated selfish nodes, but it is entirely defenseless when faced with collusion. The work in [37] provides incentives for collaboration in a network where mobile hosts are covered by several access points. The main assumption of this paper is that the hosts are selfish but not malicious. Moreover, the solution provided does not describe the underlying routing algorithm but only assumes that there is one and that it is secure. The work in [17] provides incentive to encourage packet forwarding in ad hoc wireless networks. Finally, the work in [38] focuses on preserving the anonymity and privacy of mobile hosts in a network covered by several access points.. The problem of defining compelling methods to make nodes participate in forwarding data was also addressed in the context of traditional ad hoc wireless networks [39] and resulted in designing protocols that provide fair access to the medium. However, most of the research has focused on securing routing protocols and designing key management schemes. The addressed attacks include impersonation and replay, and the solutions rely on secure association, symmetric-key-based cryptographic mechanisms, or digital signatures [40] , [41] , [42] , [43] . More sophisticated attacks such as wormhole [15] , [44] , flood-rushing [45] , or arbitrary Byzantine behavior [18] , [46] , [47] were also considered. Some of the limitations of these works are not considering colluding attackers and not working correctly in multirate networks [46] or sparse networks [47] . We note that in most of the cases, the work focused on securing specific protocols, such as DSDV [48] , DSR [49] , or AODV [26] .. A suite of stealth attacks utilizing routing information manipulation to achieve network partitioning and traffic hijacking is proposed in [50] . These attacks are not effective against JANUS because the base station is always accessible through cellular links, thus can never be removed from the routing information.. The cellular network protocols and standards also include a security component. The core services provided by any of the cellular communication protocols (such as GSM [51] , GPRS [52] , or UMTS [53] ) are authentication of subscribers (clients), providing subscriber identity confidentially, confidentiality, and integrity of both data communication and radio signaling. Clients are authenticated using a challenge-response protocol, based on a key and using algorithms stored on the Subscriber Identity Module (SIM) smart card, obtained when the client subscribes to the service.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O