Boosted multi-task learning In this paper we propose a novel algorithm for multi-task learning with boosted decision trees. We learn several different learning tasks with a joint model, explicitly addressing their commonalities through shared parameters and their differences with taskspecific ones. This enables implicit data sharing and regularization. Our algorithm is derived using the relationship between 1 -regularization and boosting. We evaluate our learning method on web-search ranking data sets from several countries. Here, multi-task learning is particularly helpful as data sets from different countries vary largely in size because of the cost of editorial judgments. Further, the proposed method obtains state-of-the-art results on a publicly available multi-task dataset. Our experiments validate that learning various tasks jointly can lead to significant improvements in performance with surprising reliability. Multi-task learning algorithms (Caruana 1997) aim to improve the performance of several learning tasks through shared models. In this paper, we introduce a novel multi-task learning algorithm for gradient boosting. This is motivated by our interest in web search ranking, where gradient boosted decision trees are state-of-the-art methods Zheng et al. 2008) .. Web search ranking is often treated as a supervised machine learning problem (Burges et al. 2005; Zheng et al. 2008) : each query-document pair is represented by a high-dimensional feature vector and its label indicates the document's degree of relevance to the query. Like many other supervised learning problems, machine learned ranking requires a large number of labeled training examples, which are time consuming and expensive to obtain. This problem becomes more acute with specialization: most major search engines offer specialized rankings for different countries or regions as shown in Fig. 1 . Queries like "football" issued in the UK generally should lead to different results than in the US. The problem of high editorial cost becomes even more prominent if one attempts to build many such specialized, country-specific ranking functions-as building each ranking function requires its own set of hand-labeled data. On the other hand, a large fraction of queries are region-insensitive. Thus, it seems worthwhile to treat the different markets as tasks that are not completely independent of one another as they share some commonalities, yet, differ enough that one cannot naïvely combine their training data sets.. There has been extensive research on multi-task learning in the past decade starting with the early works of Caruana (1997) , Thrun (1996) . Multi-task learning has been explored by various authors in different frameworks. This has lead to multi-task learning using kernel-methods (Argyriou et al. 2007 (Argyriou et al. , 2008 ; Evgeniou et al. 2006; Evgeniou and Pontil 2004) , probabilistic approaches (Bakker and Heskes 2003; Yu et al. 2005; Xue et al. 2007) , maximum-entropy discrimination (Jebara 2004) , hashing (Weinberger et al. 2009 ) as well as theoretical work (Ben-David and Schuller 2003; Maurer 2006) . However, despite the theoretical and empirical success of boosting, there are surprisingly few papers that consider multi-task learning in a boosting framework. In fact, we are not aware of any work other than ) (which combines ideas from probabilistic methods and boosting) on multi-task learning in a boosting framework.. In this paper, we propose a novel algorithm to capture task specifics and commonalities simultaneously. Given data from T different tasks, the idea is to learn T + 1 models-one for each specific task and one global model that captures the commonalities amongst them. The algorithm is derived systematically based on the connection between boosting and 1 regularization (Rosset et al. 2004 ). We are not aware of any work that jointly models several tasks by explicitly learning both the commonalities and idiosyncrasies through gradient boosted regression. Our contribution in this paper is three-fold:. 1. We introduce a novel multi-task learning algorithm based on gradient boosted decision trees-the first of its kind. 2. We exploit the connections between boosting and 1 regularization to motivate the algorithm. 3. Driven by the success of gradient boosted decision trees on web-search, we perform a detailed evaluation on web-scale datasets.. The rest of the paper is organized as follows. After reviewing related work in Sect. 2, we formally introduce the multi-task learning problem in Sect. 3 and propose our approach in Sect. 4. The connection between boosting and 1 regularization serves as motivation for our derivations. The details of machine learning for ranking are given in Sect. 5. Then in Sects. 6 and 7, we evaluate our method on several real-world large scale web-search ranking data sets from different countries. Although we mainly focus on multi-task learning for ranking across different countries, we show in Sect. 8 that our multi-task algorithm can also be used for customizing ranking functions for different query types; and also experimental results on a public dataset for regression are reported in Sect. 9. In this paper we introduced a novel multi-task learning algorithm for gradient boosted decision trees and applied it to web search ranking. We mainly focused on the problem of learning the ranking functions for various countries in a multi-task learning framework. The customization of the ranking models to each country happens naturally by modeling both the characteristics of the local countries and the commonalities separately. We provided a thorough evaluation of multi-task web search ranking on large scale real world data. Our multi-task learning method lead to reliable improvements in DCG, especially after specifically selecting sub-sets that are learned jointly.. As future work, we could look into modeling more diverse interactions between different countries. In particular, our approach is by no means restricted to a single global model. In fact, given domain-knowledge, it could make sense to add additional functions that capture the interactions between a sub-set of the countries. For example, one could imagine functions F T +1 , . . . , F T +c for various continents, such that the relevance of a document of task t , from continent c, is predicted with F 0 (x) + F t (x) + F T +c (x). Even finer interactions could be modeled through hierarchies of additional functions.. Even though our primary interest was to learn ranking functions across countries, our framework is very general and can be applied to other machine learning tasks beyond ranking for which boosted decision trees are well suited. We illustrated this point by applying our algorithm to a standard multi-task benchmark dataset. A lower bound on the dot product can be found using Hölder's inequality:. where we made use of (21) and (22) for the last inequality. Let Δβ t j = if j =j and t =t , 0 otherwise. This Δβ satisfies the 1 constraint and is also optimal because it satisfies the above lower bound with equality.. Coming back to the original variables, the optimal Δβ is thus given by (14). A common related line of research to multi-task learning is domain adaptation (DA). Here, one assumes a source domain with large training data and a target domain with very little training data. The test case is exclusively in the target domain. The main principle behind DA is to learn a model for the source and adapt it to the target domain. In the web-search example the source could be a well established country and the target a new country where the search engine is still relatively new. Gao et al. (2009) address this particular case with boosted decision trees through model interpolation and refinement. Although the authors are motivated by a very similar problem-adapting search engines to other countries-they choose a fundamentally different approach from us. While we train many functions (including a shared model) at the same time, they use an additive approach in which they first learn a base function and then train only one country-specific function.. Also related is the work on EM based multi-task boosting for face verification ). Even though this work proposes a multi-task learning framework based on boosting, it is significantly different from our approach. With data from m tasks, it learns k ≤ m boosted classifiers. A multinomial variable indicates how well each classifier performs on each task. The proposed method uses an EM algorithm to maximize the log-likelihood (based on a probability model) where the M-step involves gradient boosting.. Another approach to multi-task learning is by assuming that the tasks share a common underlying representation. This viewpoint has lead to algorithms that learn a classifier, as well as a joint shared representation simultaneously (Argyriou et al. 2007; Chen et al. 2009 ). In a sense, multi-boost does this already; the mapping given by the weak learner is a common feature representation. A function is then learned on top of such shared representation. Notation and setup Assume that we are given learning tasks t ∈ {1, 2, . . . , T }. Further, the data for these tasks, {(x 1 , y 1 ), . . . , (x n , y n )}, is also given to us. Each task t is associated with a set of indices I t that denotes the data for this particular task. These index sets form a partition of {1, . . . , n} (i.e., I t ∩ I s = ∅ when t = s and . . . , n}) . We also define I 0 = {1, . . . , n}. At this point, we assume that all the tasks share the same feature space; we will later show how this assumption can be relaxed. Finally, we suppose that we are given a cost function C t defined as a function of the predicted values for all points in I t . For instance, in a regression setting, we might consider the squared loss:. We also overload the definition of C t to allow it to be defined as a function of the parameters of the function to be learned. For instance, in case of a linear class of functions,. Previous work Previous work has mainly focused on Neural Networks (Caruana 1997; Collobert and Weston 2008) or Support Vector Machines (Evgeniou and Pontil 2004) . This latter work inspired the algorithm presented in this paper. The authors adapt SVMs to multitask learning by associating one classifier w t specifically for each task t . In addition, there is a global classifier w 0 that captures what is common among all the tasks. The joint optimization problem is then to minimize the following cost:. In Evgeniou and Pontil (2004) , all the λ i , i ≥ 1 have the same value, but λ 0 can be different. Also, a classification task is considered: the labels are y i ∈ {±1} and the loss function is. Note that the relative value between λ 0 and the other λ i controls the strength of the connection between the tasks. In the extreme case, if λ 0 → +∞, then w 0 = 0 and all tasks are decoupled; on the other hand, when λ i → +∞, ∀i ≥ 1, we obtain w i = 0 and all the tasks share the same decision function with weights w 0 .. Kernel-trick In practice, the formulation (3) suffers from the draw-back that it only allows linear decision boundaries. This can be very limiting especially for more complex real-world problems. A standard method to avoid this limitation is to apply the kernel-trick (Schölkopf and Smola 2002) and map the input vectors indirectly into a high dimensional feature space,. , where, with careful choice of φ, the data is often linearly separable. The kerneltrick is particularly powerful, because the mapping φ is implicitly chosen such that the innerproduct between two vectors φ(x i ) φ(x j ) can be pre-computed very efficiently-even if the dimensionality of φ(x i ) is very high or infinite. The kernel-trick has been adapted to multitask learning by Evgeniou and Pontil (2004) , Caponnetto et al. (2008) . Unfortunately, for many real-world problems, the quadratic time and space complexity on the number of input vectors is often prohibitive.. Hashing-trick In certain domains, such as text classification, it can be the case that the data is indeed linearly separable-even without the application of the kernel-trick. However, the input space X is often already so high dimensional, that the dense weight vectors w t become too large for learning to be feasible-especially when the number of tasks T becomes very large. Recently Weinberger et al. (2009) applied the hashing-trick to a non-regularized variation of (3) and mapped the input data of all tasks into a single lower dimensional feature space. Similar to the kernel-trick, the high-dimensional representation is never computed explicitly and all learning happens in the compact representation.

O B I O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O B I B O O O B I O O O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O B O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O B I B O O O O O O O O O B O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A distributed traffic control scheme based on edge-centric resource management I like this paper since it brings together a number of strands of the work in the problem space of Signaling and resource control in networks in a new combination, and serves to make one think about the area in a clear way, without over-selling the approach.Since the demise of integrated service, the evolution of QoS has largely been more and more towards a loose set of mechanisms under the banner of differentiated service. Within this area, core-stateless approaches have been shown by various modelling and simulation experts to provide reasonable utilisation with good guarantees, for realistic scenarios, and we now even have deployment of networks based on this (largely initially aimed at corporate VPN and VOIP deployment, but definitely successful). These networks have, to date, been typically provisioned for each set of users in a service class. We do not usually see explicit Signaling. When Signaling has been proposed, one then resorts to a number of fairly complex inter-locking mechanisms-originally, RSVP was part of the Integrated Services suite, and later adopted as one of a number of protocols to convey needs in more general scenarios, along aide either extensions to intra-domain routing protocols (OSPF-TE) or via the level 2.5 underlay, MPLS related controls.In the trend towards further "lightness of touch", researchers have looked at removing explicit user-network-interface protocols, and have attempted to distribute the Signaling and admission control functions orthogonal to the forwarding plane (albeit with links, necessarily back to the forwarding devices to ascertain current available capacity, and so on). The two main approaches discussed in this paper , and to which the authors of this paper contrast their third approach, are the Bandwidth Broker, and so-called probe-based admission control.The bandwidth broker is a well known quasi-centralised agent that edge-systems request resources from, and forwarding nodes report conditions to. The probe-based admission control schemes are fully decentralised, but suffer from lack of timely information about circumstances within the network, and indeed also, about the other ongoing probes.These authors turn this on its head, and propose a mechanism for the network to push information to the edges about ongoing resource availability, through a multicast heartbeat. This then allows edge to make self-admission control decisions more quickly, and more accurately. The scheme is evaluated through simulation and compared to the aforesaid BB and GRIP systems.The authors show that by pushing information to the edge, the traffic control can indeed be effective. I might refer to this as the difference between being told that the "bottle is half full" rather than asking "is the bottle half empty?" Multicast is also fairly widely deployed within many ISPs, although not as a user-plane service, so it is plausible that one could actually deploy the proposed scheme.Clearly, this is early work, and more evaluation in dynamic situations (e.g. during rapid route churn) plus some handle on any security problems would be good additional contributions to follow this, as well as the authors own suggestions for future work, which I hope they pursue. Public review written byJon CrowcroftThe correct admission of flows in the Differentiated Services (DiffServ) environment is critical to provide stable and predictable quality of service (QoS) to the end user. Without a scalable and precise admission control scheme, the service provider is faced with either over-provisioning the network or accepting periods of best-effort like behavior. In this paper, we propose a novel approach for admission control that exploits the unique architectural aspects of DiffServ. Through the use of periodic heartbeats emanating from edge routers to probe the network state on the available egress paths, edge routers are able to quickly conduct admission control with a tunable degree of precision. In this paper, we detail our approach, Edge-centric Resource Management (ERM), and conduct detailed simulation studies regarding the effectiveness of the approach. Over the last decade, the concept of quality of service (QoS) has evolved dramatically. Starting from the unsuccessful origins in the type of service byte (ToS) in the initial IPv4 specification, the networking research community has explored a wide variety of techniques ranging from per-flow QoS in Integrated Services [17] to aggregate or per-class QoS in the Differentiated Services model [16] . A critical aspect of each of the QoS schemes beyond best effort is the notion of traffic control, controlling what is admitted (admission control) and verifying the traffic stays within its allocation (policing).. The dominant factors for an admission control scheme are that the scheme should be fast (low setup latency), scalable (large network, large number of flows), and precise (efficient and controllable resource allocation). Hence, a wide variety of proposals have emerged to address the issues of resource management (and implicitly admission control) in the context of both inter-domain and intra-domain traffic. Efforts in the inter-domain space have built on RSVP [9] , the emerging work by the IETF NSIS (Next Step In Signaling) workgroup, and other protocols [18] . Specifically, this paper is interested in addressing the issue of intra-domain admission control within the context of one of the more promising QoS models, Differentiated Services (DiffServ).. While there has been a wide variety of efforts with regards to DiffServ QoS admission control [1, [4] [5] [6] [7] [12] [13] [14] , the majority of the efforts can be categorized into one of two categories, namely distributed approaches relying on probing and centralized approaches relying on the concept of a Bandwidth Broker (BB). In the distributed approach, nodes probe the network and make decisions for admission based on the QoS of the probe packets. Nodes act largely independently of one another relying on observed behavior rather than explicit reservation to assess the free capacity of the network. Thus, one must typically over-provision the network so as to allow appropriate safeguard regarding the imprecision of the probe data.. In contrast, centralized approaches rely on a Bandwidth Broker (BB) whereby all new connections must be approved through the BB. Provided that all policing is done correctly, no probing is necessary as the BB provides a centralized location for the domain-wise resources. Although the centralized approach can offer more precise resource allocation, it suffers from scalability constraints even when considered only on a domain-wise basis. Furthermore, both approaches can suffer from latency penalties due to either the probe/response delay or the edge router/BB delay. While such an impact may be minimal across a single domain or AS, the compound effect of multiple domains can be significant. In summary, we presented in this paper a novel approach for DiffServ admission control that incorporated the use of coordinated heartbeats coupled with the concept of a distributed fair capacity measurement. We proposed intuitive methods for capturing the inherent lagging factor associated with heartbeats and scaling factors for further governing the overall system change rate. The ERM protocol captured the implicit trade-offs associated with heartbeat scalability and addressed methods to improve both heartbeat and admission control scalability and efficiency.. To evaluate the effectiveness of ERM, we compared ERM with the traditional DiffServ Bandwidth Broker scheme, GRIP, and EAC under a wide variety of network traffic conditions. The end result from the simulations showed that ERM could offer both controllability and efficiency, all in a scalable manner. Thus, we believe ERM offers a promising new approach that delivers a fast, scalable, and precise admission control for Differentiated Services.. With regards to future work, there are several issues that merit additional study. These issues include the expansion of fairness models, derivations of formal theoretical control models for ERM, and considerations for inter-domain admission control. Thus, a service provider is faced with a dilemma. Should the provider emphasize scalability and employ a probing approach or should the provider embrace a centralized approach for additional efficiency at the cost of scalability? We believe that a new approach that offers the scalability of probing with the efficiency of centralization would offer an extremely compelling approach to admission control. In this paper, we propose a distributed scheme, ERM (Edgebased Resource Management), that offers a new approach for DiffServ intra-domain admission control.. Rather than edge routers operating independently, we introduce the notion of cooperative state gathering heartbeats that provide a relatively precise view of the network. ERM builds on the notion of a heartbeat with the idea of effective remaining fair capacity to govern the overall rate of admission control and keep the network core stable. Through simulation studies, we demonstrate how our scheme provides an intuitive and effective approach for admission control that meets the three primary goals of speed, scalability, and precision under a variety of network conditions. The remainder of our paper is organized as follows. Section 2 comments on related work and differentiates the novelty of our work. Section 3 presents an overview of ERM and discusses the network intelligence (heartbeat) aspects of ERM. Section 4 continues by discussing the admission control aspects of ERM and presenting advanced issues regarding ERM. Next, Section 5 details our simulation studies on ERM and Section 6 concludes the paper with several remarks.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O B I O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O B I O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O


Points-to analysis using BDDs This paper reports on a new approach to solving a subset-based points-to analysis for Java using Binary Decision Diagrams (BDDs). In the model checking community, BDDs have been shown very effective for representing large sets and solving very large verification problems. Our work shows that BDDs can also be very effective for developing a points-to analysis that is simple to implement and that scales well, in both space and time, to large programs.The paper first introduces BDDs and operations on BDDs using some simple points-to examples. Then, a complete subset-based points-to algorithm is presented, expressed completely using BDDs and BDD operations. This algorithm is then refined by finding appropriate variable orderings and by making the algorithm propagate sets incrementally, in order to arrive at a very efficient algorithm. Experimental results are given to justify the choice of variable ordering, to demonstrate the improvement due to incrementalization, and to compare the performance of the BDD-based solver to an efficient hand-coded graph-based solver. Finally, based on the results of the BDD-based solver, a variety of BDD-based queries are presented, including the points-to query. In this paper, we take a well-known problem from the compiler optimization community, points-to analysis, and we show how to solve this problem efficiently using reduced ordered binary decision diagrams (ROBDDs) 1 which have been shown to be very effective in the model checking community.. Whole program analyses, such as points-to analysis, require approaches that can scale well to large programs. Two popular approaches to flow-insensitive points-to analysis have been pursued in the past, equality-based approaches like those pioneered by Steensgaard [32] , and subset-based approaches like the analysis first suggested by Andersen [1] . The subset-based approaches give more accurate results, but they also lead to greater challenges for efficient implementations [6, 10, 12, 19, 25, 28, 34] . 2 For this paper, we have chosen to implement a subset-based pointsto analysis for Java. At a very high level, one can see this problem as finding the allocation sites that reach a variable in the program. Consider an allocation statement S : a = new A();. If a variable x is used in some other part of the program, then one would like to know whether x can refer to (point-to) an object allocated at S. A key problem in developing efficient solvers for the subset-based points-to analysis is that for large programs there are many pointsto sets, and each points-to set can become very large. Often, many of these points-to sets are equal or almost equal. Several methods of representing them compactly have been studied in the past, including collapsing equivalent variables [10, 24] and designing new representations for sets [11, 17, 18] ; the BDD-based approach that we introduce in this paper is another example of such a compact representation.. Since BDDs have been shown to be effective for compactly representing large sets and for solving large state space problems like those generated in model checking, it seemed like an interesting question to see if BDDs could also be used to efficiently solve the points-to problem for Java. In particular, we wanted to examine three aspects of the BDD solution: (a) execution time for the solver, (b) memory usage, and (c) ease of specifying the points-to algorithm using a standard BDD package. In summary, our experience was that BDDs were very effective in all three aspects.. The contributions of this paper include:. • We propose and develop an efficient BDD-based algorithm for subset-based points-to analysis for Java. To our knowledge, we are the first group to successfully use BDDs to solve such an analysis efficiently.. • We provide new insights into how to make the BDD-based implementation efficient in terms of space and time. First, we used a systematic approach to find a good variable ordering for the points-to set. Second, we noted that the al-gorithm should propagate the sets incrementally, and presented an incremental version. This general idea of incrementalizing the algorithm may be useful in solving other program analysis problems using BDDs. Third, we found that specifying an analysis using high-level BDD operations allowed us to specify our analysis very compactly and it was very simple to experiment with a wide variety of algorithms. Our source code (available on our web page: http://www.sable.mcgill.ca/bdd/) contains many different variations that can be enabled by switches.. • We experimentally validated the BDD-based approach by comparing its performance to a previously existing efficient solver. For small problem sizes, we found that the time and space requirements are similar, but for larger problems, the BDD-based approach requires less memory, and scales better.. • Although we initially intended to compute only points-to sets, we found that the BDD approach leads to a solution that can be used to answer a variety of queries, of which the points-to query is only one. We suggest several possible other queries.. In future work we plan to develop this aspect of our work further.. The rest of this paper is organized as follows. In Section 2 we provide an introduction to BDDs and operations on BDDs using small examples based on the points-to problem. Given this introductory material, we then introduce our points-to algorithm and its implementation using BDDs in Section 3. Then, in Section 4, we show how to improve the performance of the algorithm by choosing the correct variable ordering and making the algorithm incremental. In Section 5 we give experimental results for our best BDD algorithm and compare its performance to a hand-coded and optimized solver based on SPARK. In Section 6 we discuss possible applications for the results of our algorithm, which includes answering points-to queries. Finally, Section 7 gives a discussion of related work and Section 8 gives conclusions and future work. In this paper, we have presented a BDD-based points-to analysis that scales very well in terms of time and space, and is very easy to implement using standard BDD packages. The motivation to use BDDs came from the fact that for large programs, the number and size of points-to sets can grow so that even well-tuned traditional representations fail to scale appropriately. BDDs have been shown to work well for large problems in the model checking community, and we wanted to see if they could be applied effectively to the points-to problem. We showed that with the appropriate tuning, a fairly simple algorithm could deliver a solver that was competitive with previously existing solvers and provided a very compact representation of points-to relationships.. It was not immediately obvious that a BDD-based approach would work for a program analysis like points-to. Although BDDs have been shown to be very effective in areas like hardware verification, program analyses face program properties that are quite different from those areas, including: 1) the problem may not be represented by LTL and CTL formulas; and 2) the analyzed object may not have many common patterns. For example, the transition system of a circuit written in CTL often exhibits regularities in the structure, which can be compactly represented in BDDs by applying good heuristics. However, before we started our work, it was not clear if the subset-inclusion relationship graph, and other data structures required for points-to analysis, had common structures that could give compactness. By systematically exploring a variety of orderings and empirically analyzing the performance, we did find an incrementalized algorithm and associated variable ordering that led to compact BDD representations. It is interesting to note that in the case of points-to analysis, it was not so important to find a compact representation for the input problem (unlike the case of hardware verification, where the input description may be very large and have many common patterns), but it was important to find a compact representation for the solution (i.e. the points-to relationships). Thus, it was the fact that the points-to sets showed a lot of regularity that leads to a fast and space-efficient solution. It would be very interesting to see if other whole-program analyses exhibit the same sort of regularity in their solutions. In our opinion, this is very likely.. In our work so far, we concentrated on choosing a good variable ordering and developing the incremental propagation algorithm. It is possible that this could be further improved by introducing some aspects of graph-based solvers into the BDD solver. For example, it would be very interesting to see if efficient BDD algorithms for collapsing strongly connected components [35] would further improve the efficiency of our BDD-based points-to algorithm. Another idea which has been suggested for improving the efficiency of BDDs is dynamic variable reordering. Our preliminary experiments with using this technique on our points-to problem showed no significant improvement. In fact, the cost of reordering the large BDDs involved appeared to be even higher than the cost of solving the points-to problem with the original ordering. We therefore leave further investigation of dynamic variable reordering for BDD-based points-to analysis as future work.. In addition to achieving our goals in terms of time and space, we were pleasantly surprised with how easy it was for us to specify a wide variety of algorithms with the BDD approach. We tried many variations of the points-to analysis while developing our algorithm and it was very easy to go from one variation to the next. Based on this experience, we believe that a BDD package should be part of the standard toolkit for compiler analysis developers. Further, our BDD-based points-to analysis should be very easy to incorporate into program analysis tools where BDDs are used more and more frequently.. We plan to continue our work with BDDs and to further experiment with the kinds of queries outlined in Section 6. In addition, we would like to make a tighter connection between the Soot frame-work and a BDD toolkit so that subsequent BDD-based analyses could be specified at a very high level. Points-to analysis [1, 9, 32] has been an active research field in the past several years. Hind [14] gives a very good overview of the current state of algorithms and metrics for points-to analysis. An important issue is how well the algorithms scale to large programs. Various points-to analyses make trade-offs between efficiency and precision. Equality-based analysis [32] runs in almost linear time, but with less precise results. On the other hand, subset-based analysis [1] produces more precise results, but with cubic worst-case complexity. In this work, we developed a specific version of a subset-based analysis that is suitable for implementing with BDDs, and which exhibits good space and time behaviour when used to analyze a range of Java programs, including a variety of large benchmark programs.. Various optimizations have been proposed to improve the efficiency of points-to analyses. Two of these optimizations, cycle elimination [10] and variable substitution [24] , are based on the idea that variables whose points-to sets are provably equal can be merged, so that a single representation of the set can be shared by multiple variables. Heintze and Tardieu [12] reported very fast analysis times using a demand-driven algorithm and a carefully designed implementation of points-to sets [11] .. Several groups adapted the points-to analyses used for C to Java [19, 25] . These approaches, however, were applied only to benchmarks using the JDK 1.1.8 class library. One of the difficult points for whole program analysis for Java is that even very simple programs touch, or appear to touch, a large part of the class library. The JDK 1.3.1 class library is several times larger than the 1.1.8 library, and techniques which applied to the 1.1.8 case may no longer scale.. Recently, two approaches have been presented that have been shown to scale well to the JDK 1.3.1 library. Whaley and Lam [34] adapted the approach of Heintze and Tardieu to Java programs, and managed to get it to scale to benchmarks using the JDK 1.3.1 class library (although they made optimistic, potentially unsafe, assumptions about what part of the library needs to be analyzed). Lhoták and Hendren [17, 18] presented the SPARK framework, which allows experimentation with many variations of points-to analyses for Java. They used this framework to implement points-to solvers that were more efficient in time and space than the other reported work, including that of Whaley and Lam, making it the most efficient Java points-to analysis solver of which we are aware. Therefore, in our study of BDD-based points-to analysis, we used the SPARK framework both to generate the input to our BDD-based solver, and as a baseline solver against which to compare our new BDD-based solver.. Ordered Binary Decision Diagrams [5] represent boolean functions as DAGs. The canonical representation allows efficient boolean operations and testing of equivalence and satisfiability. Symbolic Model Checking [16] is used to verify circuits with an extremely large number of states by using BDDs. The use of BDDs in this context has allowed researchers to solve larger problems than can be solved using table-based representations of graphs. BDDs have also been used in software verification and program analyses. PAS [29] converts predicate and condition relations in a control flow graph to a compact BDD representation and performs analysis on these BDDs. Another use of BDDs is to represent large sets and maps; TVLA [21] and Bebop [3] are examples. In our work, as in model checking, we use BDDs to represent all data structures, and we show non-trivial techniques to make the original algorithm scalable to large programs using this new representation.. Although model checking and program analyses are not tightly connected yet, several publications have pointed out theoretical connections between them [26, 27] . The theoretical foundation of flow analyses is the fixed-point theory on monotonic functions, whose counterpart in model checking is the modal µ-calculus. Schmidt and Steffen [27] presented a methodology of treating iterative flow analysis as model checking of abstract interpretations. Bandera [8] is a tool-set applying such ideas to analyzing realistic programs. Like some other work [4] , it abstracts program properties to linear temporal logic (LTL) or computational tree logic (CTL) formulas, which can be verified efficiently by existing model checking tools. Martena and Pietro [22] studied the application of a model checker, Spin, to solve intraprocedural alias analysis for C. In a different program analysis setting, BDD-based groundness analysis for constraint (logic) programs has become one of the standard approaches [13] . We selected benchmarks from the SPECjvm98 [31] suite, SPECjbb2000 [30] and three other large benchmarks: sablecc-j, soot-c and jedit. Table 1 shows the description of each benchmark. Sablecc-j is a parser generator written in Performs multiple database functions on memory resident database. raytrace A raytracer that works on a scene depicting a dinosaur. mpegaudio Decompresses audio files that conform to the ISO MPEG Layer-3 audio specification. jack A Java parser generator that is based on the Purdue Compiler Construction Tool Set (PCCTS). jess Java Expert Shell System. sablecc-j An object-oriented parser generator written in Java. jbb2000. A Java program emulating a 3-tier system with emphasis on the middle tier. javac. The Java compiler from the JDK 1.0.2. soot-c A bytecode to bytecode optimization and annotation framework. jedit A full-featured editor written in Java. We generated the constraints for our BDD-based solver using the SPARK points-to analysis framework [17, 18] . Constraints were generated for a field-sensitive analysis (using a separate points-to set for each field of the objects allocated at each allocation site). The call graph used for interprocedural flow of pointers was constructed using class hierarchy analysis. Effects of native methods were considered, as supported by SPARK.. The raw points-to constraints generated from an input program can either be fed directly to a solver as input, or they can first be simplified off-line by substituting a single variable for groups of variables known to have the same points-to set [24] . This results in a smaller set of constraints for an equivalent problem. We used the SPARK framework to generate both unsimplified and simplified versions of the constraints as input to our BDD solver. We denote a simplified set of constraints with the letter s, and an unsimplified set of constraints with ns.. A points-to analysis solver for a typed language such as Java has two reasonable options for dealing with the declared types of variables: it can solve the points-to constraints first, and restrict the points-to sets to subtypes of the declared type afterwards [25, 33] ; alternatively, it can remove objects of incompatible type as the points-to analysis proceeds [17, 18, 34] . Both our BDD solver and the SPARK solver support both variations. We denote a solver that respects declared types throughout the analysis with the letter t, and one that ignores them until the end with nt.. The two options for simplification and two options for handling of types result in four combinations, all four of which have been used in related work. In this study, we focus on three of them: (s/t), (ns/t), and (ns/nt). We stress that the same sets of constraints were used as input to both our BDD solver and the SPARK solver.. The BDD Solver is written in C++ and uses the BuDDy 2.1 C++ interface compiled with GCC 2.95.4 at -O3. The BuDDy package has a built-in reference counting mark-and-compact garbage collector for recycling BDD nodes. Whenever the proportion of free nodes is less than a threshold (20% by default), the kernel increases the node table size. In our experiments for performance measurement, we used a heap size of 160M. All our experimental data were collected on a

O B O O O B O O O O O O O O O O B O O O B I I O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O B O O O O O O O O O O O B O O O O O B O O O O O O O O O O B O O O O O O O O O O O O B O O O O B O O O O O O B O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O B O O O O O O O O O O O B O O O O O O O O O O O O B I I O O O O O O O O O O O O B I O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B B O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O B B O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O B B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O


Linear-quadratic blind source separating structure for removing show-through in scanned documents Abstract Digital documents are usually degraded during the scanning process due to the contents of the backside of the scanned manuscript. This is often caused by the showthrough effect, i.e. the backside image that interferes with the main front side picture due to the intrinsic transparency of the paper. This phenomenon is one of the degradations that one would like to remove especially in the field of Optical Character Recognition (OCR) or document digitalization which require denoised texts as inputs. In this paper, we first propose a novel and general nonlinear model for canceling the show-through phenomenon. A nonlinear blind source separation algorithm is used for this purpose based on a new recursive and extendible structure. However, the results are restricted due to a blurring effect that appears during the scanning process due to the light transfer function of the paper. Consequently, for improving the results, we introduce a refined separating architecture for simultaneously removing the show-through and blurring effects. Archives usually need automatic methods for improving the readability of the digital versions of the ancient or printed documents. This processing is also recommended for the applications concerned with the production of the machinereadable digital document from ancient handwriting for example by using character recognition algorithms; these algorithms require clean versions of the original documents. In this paper, we consider one of the most common degradations, usually appearing in ancient documents that are written or printed on both sides of the page: this phenomenon is generally called print-through. Print-through is an undesired appearance of a printed image or text of the reverse side of the paper and can be divided into three additive components, each of them corresponding to a physical phenomenon [1] : -The show-through component related to the paper's intrinsic transparency or low thickness; -The pigment penetration component; -The vehicle oil component, which is related to the loss of opacity due to the filling of pores with oil.. When the ink of the printer does not penetrate the paper considerably, effects of pigment penetration and of vehicle oil are negligible and print-through can be approximated only by show-through. Show-through can significantly impair the readability of the document and also cause visual fatigue for the reader. When the show-through degradation is significant (the darkness of the show-through is comparable to, or even greater than, that of some parts of the desired writing), then it is practically impossible to remove show-through by only using a simple thresholding operation.. Several approaches for show-through reduction have been already investigated. In [2, 3] , the authors used various features in the document for distinguishing show-through from the foreground image and presented show-through removal techniques involving only one side of the document. Although these methods certainly perform better than simple thresholding, there is no way to unambiguously differentiate foreground from show-through without comparing both sides of the document especially in grayscale images. Other works process both sides of the document simultaneously, in order to identify regions that are mainly show-through, and replace them by an estimate of the background [4, 5] . Most of these works deal with only text or handwriting, and the original images are degraded after the show-through removal procedure. Recent investigations are applying Blind Source Separation (BSS) algorithms for solving this problem. In [6, 7] , the authors assume that the show-through effect can be modeled by linear superimposition of the back and front sides. Then, the scanned front-side image (corrupted by the backside) and the scanned back-side image (corrupted by the frontside) are linear mixtures of the pure front and back images, assumed to be independent, which can be estimated by BSS techniques. Tonazzini et al. in [8] represent an effective method for removing show-through in color images by using only one side of the paper. However, the method is not applicable for grayscale images. Although these methods give good results, the results are not perfect especially in regions where the images of the front and back sides of the paper overlap and the front side's image is nearly black. In such regions, the recovered front image is whiter than other sections where there is no overlap. The main reason of these poor results is that show-through is a nonlinear effect as we will show in the next section, and as it has also been previously considered and modeled, e.g. in [9] . Sharma in [10] considered a nonlinear model for this phenomenon and proposed to compensate for this effect by using adaptive filters. The main disadvantage of using adaptive filters, as we will see later, is that the Signal to Noise Ratio (SNR) of the outputs cannot be more than the noise to signal ratio of the input images (due to the power inversion property in adaptive filters [11, page 78] ). Castro et al. [12] used fuzzy classification for detecting degraded sections of the input images. The proposed algorithm is fully dependent on the specific application (only applicable to ancient music notes) and suffers from disadvantages of fuzzy classification. Moreover, they did not consider the blurring effect that we will point out in Sect. 4.1. By taking into account the nonlinearity of showthrough, Almeida used in [13] the MISEP method based on Multi-Layer Perceptron (MLP) networks for separating the real-world nonlinear image mixtures. The main drawback of using these kinds of "universal nonlinear networks" for BSS is the separability issue: the Independent Component Analysis (ICA) does not necessarily lead to BSS using such networks. Note that based on the way that the show-through phenomenon is modeled, this or other nonlinear BSS techniques [14] [15] [16] [17] may be applicable for solving this problem. In fact, finding a suitable BSS method for separating the show-through from the original sources based on the acquired model is one of the main goals of this paper.. Here, it is worth to mention that, although there are some Markov Random Field (MRF)-based show-through removal methods [18, 19] , in this paper, we only focus on BSS-based algorithms. This is because of the fact that in the framework of BSS, considering MRF nature of images requires one to take into account the joint distribution of neighbor pixels, which leads to very tricky estimation problems and very cost-consuming algorithms. In fact, the Markovian nature has already been considered in BSS, for instance by Hosseini et al. [20] . In addition in the framework of hyperspectral processing, it has been shown that MRF and BSS, although based on different properties, lead to very similar results [21] .. In this paper, we first show that show-through is a nonlinear phenomenon and introduce a nonlinear model based on experiments. Then, we show that, in addition to nonlinear mixing models, we have to take into account a blurring effect. Our method is derived from the nonlinear BSS recursive structure presented by Hosseini and Deville [22, 23] , in which we add refinements for jointly compensating for the degradation caused by show-through and blurring effects, as it will be demonstrated by some simulation results. Although the principles and preliminary results have briefly been presented in [24] , the current paper, both in theoretical and experimental parts, contains many new extensions.. The paper is organized as follows. In Sect. 2, we experimentally show that show-through is a nonlinear phenomenon and introduce our nonlinear model. Section 3 describes a basic blind separation structure for dealing with this nonlinear model. The blurring effect and the usage of adaptive filters for its compensation, and the combination of these filters with the non-linear BSS structure are developed in Sect. 4. Finally, a few experimental results with real and artificial printed or manuscript documents are presented in Sect. 5, before the conclusions and perspectives of Sect. 6. In this paper, we experimentally showed that show-through is a nonlinear phenomenon and can be approximately represented by a linear-quadratic model. For such a mixing model, based on [22, 23] , we first used a linear-quadratic separating structure which is able to extract the original sources. Simulations show that this structure can remove show-through but cannot cancel the blurring effect implied by the scanning process due to the paper's intrinsic transparency. Therefore, we proposed a newer adaptive linear-quadratic blind source separating structure, in which one adds adaptive filters to estimate the shape of the point spread function of the paper. Therefore, using this separating structure, one can simultaneously remove show-through and blurring effects. Finally, we justified the effectiveness of this new method with a few experiments on simulated mixtures and real-world scanned images. The results were very satisfactory in comparison with other current show-through removal methods.. The main advantages of this show-through removal method can be summarized as:. − It may be extended to more complicated polynomial models [22, 23] ; − Filter coefficients are updated adaptively. Therefore, the structure is able to estimate the point spread function of the paper and remove the blurring effect as well, without priors on the paper properties; − This structure is also applicable to linear mixtures: in this case, the parameters q 1 and q 2 will become zero.. For the near future, we are planning to complete our study about the theoretical invertibility of bilinear mixing models. In addition, investigating of the effects of those simplifying assumptions that we used in our method can be useful for improving the performance of the proposed algorithm. Another problem to be addressed is to find a procedure to evaluate the quality of the output results. By these objective performance indexes, we can provide more consistent comparisons between show-through removal algorithms.

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Multicore implementation of a fixed-complexity tree-search detector for MIMO communications Abstract Multicore systems allow the efficient implementation of signal processing algorithms for communication systems due to their high parallel processing capabilities. In this paper, we present a high-throughput multicore implementation of a fixedcomplexity tree-search-based detector interesting for MIMO wireless communication systems. Experimental results confirm that this implementation allows to accelerate the data detection stage for different constellation sizes and number of subcarriers. This work showed the potential of parallel processing to cope with the costly signal processing algorithms necessary to carry out MIMO wireless communication. An efficient multicore implementation of a fixed-complexity MIMO detector was here presented. The experimental results indicate that the proposed implementation considerably reduces the execution time required to perform data detection with respect to a single-core (sequential) CPU implementation of the same detection algorithm. Furthermore, the runtime of the proposed approach fulfills the requirements of current wireless communication standards for some configurations. Therefore, the high throughput and ability to reconfigure features clearly evidence that multicore systems are meaningful to develop versatile and low-cost SDR platforms.

B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O


Default Bayesian model determination methods for generalised linear mixed models In this paper, we consider a default strategy for fully Bayesian model determination for GLMMs.We address the two key issues of default prior specification and computation. In particular, we extend a concept of unit information to the priors for the parameters of a GLMM. We rely on a combination of MCMC and Laplace approximations to compute approximations to the posterior model probabilities and then further approximate these posterior model probabilities using bridge sampling. We apply our strategy to two examples. Generalised linear mixed models (GLMMs) extend generalised linear models (GLMs) to responses which are correlated due to the existence of groups or clusters, by the inclusion of group-specific parameters (known as random effects in classical statistics). For example, in a longitudinal study we record several observations from the same individual. GLMs, linear mixed models (LMMs), and linear models (LMs) are all special cases of GLMMs. In this paper, we considered a default strategy for model determination amongst GLMMs under weak prior information and where the dispersion parameter of the exponential family is unknown. Our strategy takes into account default prior specification for the regression parameters and the variance components, and describes a general computational strategy.. The default priors are based on a unit information concept that has proved successful for other authors.. The general computational strategy is based on two phases. Phase one combines a Laplace approximation of the integrated likelihood with an MCMC method to findf L (m|y); an approximation to the posterior model probabilities. Thesef L (m|y) are then used to define M a candidate set of promising models on which to focus. Phase two involves performing the more computationally expensive but more accurate bridge sampling on the models in M to findf BS (m|y).

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O


Connected Equi-Length Line Segments for Curve and Structure Matching Abstract Matching of two images is one of the all pervading problems in computer vision. It is a key step in 2D and 3D object recognition from 2D and 3D image description. It is fundamental to image fusion, registration and many other problems. In essence, the matching problem involves a collection of object primitives or features extracted from the images, such as edges, boundaries, special structures and junctions. Boundaries are the most frequently used features for matching because they represent much of the intrinsic structures of an image. [16] . A reliable representation of curves, suitable for matching, should be invariant to rotation, scaling, and translation to make recognition of the curve possible after arbitrary instances of those transformations.. Medioni and Nevatia [17] cast the problem of shape matching as a line segment matching problem. An object contour is obtained by some edge detection method from which features such as the length of a segment, coordinates of the end points, orientation and average contrast are computed. The sum of the weighted absolute differences of the feature values between a model and a scene segment is the shape measure between the two segments. This measure indicates the goodness of match between the two segments. A stochastic labeling scheme is then used to label each model segment either as one of the scene segments. Since the method uses relaxation labeling, it is computationally intensive. A good estimate of the initial assignment of the labels is important relative to the convergence of the algorithm and the validity of the result. Enhancements to improve the performance to relaxation labeling can be found in [18] and [19] . The technique is sensitive to scale variation because the feature value, such as the length of a line segment, used in this technique is inherently scale dependent.. Other approaches use critical points [5] and landmarks [10] to represent objects. The landmarks of an object are points of interest relative to the object that have important shape attributes, such as corners, holes, protrusions and high curvature points. A landmark detector detects and orders the landmarks in sequence that corresponds to consecutive points along the object boundary. A measure of similarity of between two landmarks, one from the model and the other from the scene, is needed to perform this matching. The merit of landmark-based shape recognition is that the extraction of the entire object contour is not required to achieve recognition and consequently it considerably saves computation time. However not all objects have salient key points (think of a circle for example) and using the key points alone sacrifices the shape information available in the smoother portions of the object contours.. Another class of methods try to find hierarchical straight line or angle approximations to curve, including strip tree [6] , arc tree [13] [12] and HAL tree [3] , etc. The strip tree scheme splits recursively a curve segment into two sub-curve segments at the curve point that has the maximum approximation error from the approximated line segment for the curve segment. A rectangle, which contains all of the points on a segment, is used to represent the segment. The algorithm stops when every point on the curve is within some threshold distance from the line segment. However, this method is not robust enough and the representation can change greatly due to a small change in the curve. Moreover, if the curve is closed, an arbitrary choice of end points must be made which will certainly affect the representation.. We propose in this paper a new technique that uses Connected Equi-Length Line Segment (CELLS) as a model for curve representation and matching to overcome the disadvantages of the existing methods. Our technique was originally motivated for applications in the domain of aerial images, but the method is of general utility. The idea of breaking a curve into several line segments to represent it is not new, and several traditional methods, such as chain code and polygonal approximation, involve this concept. However, these techniques are sensitive to scale variation because the features, such as the length of a line segment, used in these techniques is inherently scale dependent. The CELLS method adopts a new idea to overcome this shortcoming. Furthermore, piecewise linear approximation contains as much as information of a curve as possible. As the number of the line segments increases, detailed portions of the curve, such as the dominate points and the smoothing parts, can be described arbitrarily accurate. A group of line segments is a very good descriptor to represent the features of the curve and is suitable for matching.. Our primary contribution is to represent a curve by a number of orientated line segments of equal length. Each line segment has a head and tail, pointing from the head to tail. Each line segment is attached with an identification vector, which reflects the distribution of the rest of line segments with respect to the current one using orientation difference. A new matrix called Orientation Difference Matrix (ODM) is constructed from the identification vectors. We develop a difference measure that measures the distances between ODMs to estimate if the two curves are well matched. For a group of curves in the model and scene, the number of line segments is the same to guarantee the representation is scale invariant. An area criterion is applied to determine this number. The area criterion ensures the unique representation of each curve in the scene has the unique representation as well as the reduction of the computational complexity. The most attractive property of the CELLS method is that it easily obtains a representation that is inherently invariant to rotation, scaling and translation. It is robust and small perturbation will not affect the representation. The matching of structures consisting multiple interwoven curves is also considered in this paper. The structure matching is based on CELLS. The junctions on the structure, including tri-junctions and quad-junctions, are identified first. We model each arch on the structure using CELLS. The structure matching is converted into a problem of junction matching. We compare two junctions by measuring the difference of each group of arches linking the same junction. As the proposed method is named CELLS, we term each line segment as a cell. A cell sequence is generated when we use CELLS to represent a curve. This paper is organized as follows. In Section 2, we describe in detail the novel CELLS method, and study its computational complexity. We analyze some properties of the proposed method, such as invariances and uniqueness in Section 3. The problem of structure matching is addressed in Section 4. The task of registration of SAR images to maps is discussed in Section 5, and the experimental results will also be presented there. and ) , for . We neglect the elements in the lower left triangle of the matrix because according to Equation (5), it satisfies the conditions that 1). X axis to the cells on two curves. After normalization of the cell sequences, the following expressions are tenable 1) , and 2). . As , we obtain , and consequently. , . It means that with respect to the first cells of the cell sequences, all of the other corresponding pair of cells have the same position relative to the first one. Therefore, by contra proposition, two cell sequences have no difference in shape, that is, CS . It has been proved that different cell sequences have different ODMs. . , we use the CELLS method to model it and obtain a cell sequence. , is invariant to translation.. Proof: Let us first consider the task to shift a curve, with cell sequence and. , to a new location by using displacement ( )

O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


New constructions of OSBE schemes and their applications in oblivious access control Abstract Oblivious signature-based envelope (OSBE) schemes have demonstrated their potential applications in the protection of users privacy and rights. In an OSBE protocol, an encrypted message can only be decrypted by the receiver who holds a valid signature on a public message, while the sender (encrypter) does not know whether the receiver has the signature or not. Our major contributions in this work lie in the following aspects. We improve the notion of OSBE so that a valid credential holder cannot share his/her credential with other users (i.e., all-or-nothing non-transferability). We clarify the relationship between one-round OSBE and identity-based encryption (IBE) and show that one-round OSBE and semantically secure IBE against the adaptively chosen identity attack (IND-ID-CPA) are equivalent, if the signature in the OSBE scheme is existentially unforgeable against adaptively chosen message attacks. We propose an oblivious access control scheme to protect user privacy without the aid of any zero-knowledge proof. Finally, we also highlight some other novel applications of OSBE, such as attributed-based encryption. Internet services usually require users to provide their personally identifiable information (PII) for registration. Frequent registrations using PII could make the user traceable or even impersonated. Therefore, how to protect users' PII has become increasingly important. Aiming to protect users' privacy, some elegant solutions have been proposed, for example, privacy-enhancement scheme [1, 2] , privacy-preserving schemes [3, 4] , user-centric privacy management scheme [5] , identity management scheme [6] , etc.. Introduced by Li et al. [7] , oblivious signature-based envelope (OSBE) is a protocol, where a sender sends an envelope, which encapsulates a secret, to a receiver, so that only the receiver who has obtained a valid signature on an agreedupon message from the certificate authority (CA) can open the envelope and then obtain the secret. Further, the sender does not know whether the receiver has obtained the required signature. Hence, OSBE provides oblivious property for the receiver about the signature. OSBE is especially efficient in handing the cyclic policy interdependency problem in automated trust negotiation (ATN) [8] . For example, both Alice and Bob are agents of Central Intelligence Agency (CIA). They can only communicate with other CIA agents. Consider the problem of creating a secure session between Alice and Bob. Due to their special identities, none of them wants to release their identities first. Therefore, no session can be created. OSBE can deal with this scenario efficiently. If Alice wants to initiate a session with Bob, she can send an envelope, which encapsulates her identity, such as her public key certificate, to Bob. If Bob has possessed a credential (signature on the message which can indicate the membership of CIA) from CIA, he can use it to open the envelope and obtain Alice's identity. Otherwise, nothing about Alice's identity is released to Bob.. Li et al. [7] proposed three OSBE schemes based on Rivest [9] , Boneh [10] and Rabin [11] signatures, respectively. The first one requires a two-round communication, while the rest are obtained from identity-based encryption (IBE) [12, 13] and require one-round. They pointed out that OSBE can be used in some schemes, such as ATN [8] , two-party secure function evaluation (SFE) and complicated policy requirements [14] . They also left an open problem that how to design an efficient and provably secure OSBE scheme from a DSA signature. They envisioned that OSBE can be used to construct oblivious access control (OAC) systems. The idea of OSBE has been also used to constructed secret handshake [15] , oblivious certificates and oblivious credentials [16, 17] .. Nasserian and Tsudik [18] revisited OSBE and solved the problem raised in [7] . They proposed several OSBE schemes from ElGamal signature family, such as Schnorr signature, Nyberg-Ruppel signature, DSA signature, et al. They also pointed out that OSBE can be used in Blogs and peer-to-peer (P2P) networks.. Proposed by Holt et al. [19] , hidden credentials are important primitives to protect services, access control policy and extremely sensitive private information. In a hidden credential scheme, a sender encrypts a secret and sends it to a receiver. The receiver must possess the required credential(s) in order to decrypt it. Hidden credentials have been used to conceal complex policies [20] and hide attributes in access control [4] . In these schemes, a receiver can access the resources if and only if his attributes satisfy the policy specified by the sender, while the sender knows nothing about the user's credentials. Moreover, he does not know whether the receiver has a credential.. Coull et al. [21] proposed a protocol for anonymous access to an oblivious database using stateful anonymous credentials. Here, a stateful graph was used. With each access, the user's state was transformed from one to another. For each state, the user must possess a corresponding credential from the credential provider. If all states were used, the user could not access the database again. Camenisch et al. [22] proposed another protocol for anonymous access to a database, which avoided reissuing credentials and was more efficient than that of [21] . In both schemes mentioned above, two building blocks were used, namely anonymous credential and oblivious transfer (OT). A user must obtain the required credentials from the issuer prior to access the database and then prove them to the database server in zero knowledge. The database server and the user execute an OT protocol to deliver the services selected by the user. As a result, the following features are captured: (1) Only the authorized users can access the database; (2) The database server knows nothing about the user's attributes; and (3) The database server does not know which services the user can access. In this paper, we clarified the relationship between the one-round OSBE and IBE. We proposed two novel OSBE schemes based on BB signature and MBBS signature. The credentials in our MBBS-OSBE scheme are all-or-nothing non-transferable. Based on our MBBS-OSBE scheme and an OT scheme, we proposed a new access control scheme called OAC. In our OAC scheme, legal users can obliviously obtain services from the service providers. As a result, the user does not release anything about his credential and the selected services to the service providers, while there are no aid of zero-knowledge proof. We also pointed out the potential applications of OSBE in ABE.

O O O O O O O O O O B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O