Application-specific service technologies for commodity operating systems in real-time environments In order to eliminate the costs of proprietary systems and special purpose hardware, many real-time and embedded computing platforms are being built on commodity operating systems and generic hardware. Unfortunately, many such systems are ill-suited to the low-latency and predictable timing requirements of real-time applications. This article, therefore, focuses on application-specific service technologies for low-cost commodity operating systems and hardware, so that real-time service guarantees can be met. We describe contrasting methods to deploy first-class services on commodity systems that are dispatched with low latency and execute asynchronously according to bounds on CPU, memory, and I/O device usage. Specifically, we present a "user-level sandboxing" (ULS) mechanism that relies on hardware protection to isolate applicationspecific services from the core kernel. This approach is compared with a hybrid language and runtime protection scheme, called SafeX, that allows untrusted services to be dynamically linked and loaded into a base kernel. SafeX and ULS have been implemented on commodity Linux systems. Experimental results have shown-that both approaches are capable of reducing service violations (and, hence, better qualities of service) for real-time tasks, compared to traditional user-level methods of service deployment in processprivate address spaces. ULS imposes minimal additional overheads on service dispatch latency compared to SafeX, with the advantage that it does not require application-specific services to execute in the trusted kernel domain. As evidence of the potential capabilities of ULS, we show how a user-level networking stack can be implemented to avoid data copying via the kernel and allow packet processing without explicit process scheduling. This improves throughput and reduces jitter. Recent trends have seen the use of commercial off-the-shelf (COTS) systems and hardware being deployed in real-time and embedded computing environments. For example, systems such as Linux are now being deployed in real-time and embedded settings [Yodaiken and Barabanov 1997] . Not only does this lead to a reduction in development and operation costs but also it enables a common code base for systems software to be used in both special-and general-purpose computing. However, COTS systems (e.g., Authors' addresses: R. West and G. Parmer, Computer Science Department, Boston University, 111 Cummington Street, Boston, MA 02215. email: {richwest, gabep1}@cs.bu.edu. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested fromLinux) are typically ill suited to the needs of specific applications, especially when they must operate with real-time requirements.. The protection afforded by commodity operating systems usually restricts applications to process-private address spaces via which system calls can be made to access more privileged services of the trusted kernel. While the process model has many virtues, it incurs significant overheads due to scheduling, context switching, and interprocess communication. Similarly, there is often a "semantic gap" between the requirements of code that executes at user-level and the interface via which requests are made for kernel-level services. For example, to implement a real-time monitoring and control application may require one or more processes to be executed periodically and/or may involve various tasks to respond to asynchronous events (e.g., hardware interrupts) with bounded latency. Using traditional system calls to establish handlers for kernel events (e.g., via signals) and to ensure processes execute within specific deadlines is cumbersome at best, but more typically does not guarantee the necessary responsiveness of real-time applications. For this reason, we have been developing various mechanisms to support application-specific services on commodity operating systems that can be activated with low latency and executed according to strict timing requirements without the need for scheduling and context-switching between process-private address spaces.. In our original work on support for application-specific service extensions of commodity operating systems, we developed SafeX [West and Gloudon 2002] . SafeX is a hybrid language and runtime approach, supporting kernel-level service extensions with quality-of-service (QoS) requirements. Extensions are written in a type-safe language and restricted on the range of memory addresses they may access. By dynamically linking them into a running kernel, they can be used to affect service management decisions, by monitoring and adapting resource usage on behalf of specific applications.. While SafeX enables applications to bridge the semantic gap between their needs and the provisions of the underlying system, it conflicts with one of the basic philosophies of good system design. For many years, system designers have considered the idea of a kernel to be the address space in which only the most trusted and fundamental services should reside. For this reason, we have taken lessons learned from SafeX to develop a new mechanism for deploying "first-class" application-specific services and handlers at the user level. The idea behind first-class user-level services is to grant them (where possible) the same privileges and capabilities of kernel services, with the exception that the kernel may revoke access rights to any services abusing their privileges.. With this vision in mind, this article compares our user-level sandboxing (ULS) scheme against SafeX for the purpose of implementing real-time and asynchronous services and handlers on commodity operating systems such as Linux. We show how user-level services may be dispatched with almost the same latency as kernel-level interrupt handlers [Wallach et al. 1997] , while also being executed without scheduling and context-switching overheads associated with processes. In fact, both SafeX and ULS ensure that service extensions are invoked (when necessary) without being at the mercy of kernel-level scheduling policies that are inherently non-real-time, or which may result in unbounded delays. For example, in many traditional systems, user-level processes may register signal handlers to be invoked when specific kernel events occur, but these handlers only run when the corresponding process is scheduled and that may be after an arbitrary amount of time.. In contrast, our approaches enable an application process, P i , to register a firstclass handler that responds to for example, timer interrupts without P i having to execute. Given our ability to bound the dispatch latency of application-specific services, we show empirically the improved service guarantees and reduced violation rates of both ULS and SafeX compared with alternative user-level methods of implementing application-specific services. Specifically, we compare user-and kernel-level implementations of a feedback-control service, for managing the allocation of CPU cycles to application processes according to their resource requirements over finite windows of real time. Such a service might be beneficial to multimedia applications requiring specific CPU shares at designated time intervals to encode/decode audio and video streams. Alternatively, a control application may wish to guarantee that the correct share of CPU time is available to process sensor readings in a timely fashion.. From experiments, we show that ULS and SafeX are low-cost mechanisms for the timely and predictable execution of application-configurable services and handlers on commodity operating systems. A series of adaptive CPU service management tests on Linux shows that ULS handlers and SafeX kernel extensions can reduce deadline miss rates by a factor of 4, compared to process/thread-based methods of service execution. ULS and SafeX bridge the gap between the agnostic services of general-purpose systems and the needs of individual applications, including those with real-time requirements. Unlike SafeX, ULS does not require the core kernel to be polluted with potentially unsafe code that may jeopardize the integrity of the system and, therefore, its ability to meet service guarantees. The potential capabilities of ULS are highlighted with the implementation of a user-space networking stack that improves throughput and reduces jitter when packet forwarding, without the need for special hardware support.. In the next section we provide a brief overview of our prior work on SafeX, followed in Section 3 by further details about the ULS method to deploy first-class services. The performance benefits of ULS and SafeX are evaluated empirically in Section 4. Related work is discussed in Section 5, while conclusions and future work are outlined in Section 6. This article compares various methods to instrument commodity operating systems with services and handlers that are tailored to the needs of real-time applications. We compare methods to deploy service extensions at both the kernel and user levels, using our SafeX and user-level sandboxing (ULS) schemes, respectively. Both approaches enable applications to deploy services in a manner that does not require explicit scheduling and context switching between process-private address spaces, thereby ensuring bounded dispatch latencies and finer-grained resource management. SafeX relies on a combination of type-safe language and runtime support to enforce memory, CPU, and I/O-space protection of untrusted application-specific services within the address space of the trusted kernel. This enables users to deploy "first-class" services having the same capabilities as core kernel services, with the exception that the kernel may revoke access rights on any services abusing their privileges. Such an approach prevents, for example, an application-specific service from running for unbounded amounts of time and/or altering its resource usage beyond that allowed by the kernel. However, SafeX places restrictions on how extensions operate within the kernel by preventing them from disabling interrupts and accessing kernel symbols outside those within a defined API.. To alleviate the potential problems associated with application-specific services executing in the trusted kernel address space, our ULS approach allows first-class services to execute in a sandbox environment isolated by hardware-level (i.e., page-based) protection from the kernel. This imposes only minimal additional overheads over those associated with SafeX when dispatching application services. Specifically, ULS requires an upcall into a sandboxed memory region as well as a TLB flush of a single page entry. Such a cost is minimal and bounded compared to the overheads of otherwise scheduling and context switching between user-level process address spaces.. Experimental results show that ULS and SafeX incur similar performance penalties and benefits for an example service extension that adaptively manages CPU usage among competing processes in specific windows of real time. Both approaches yield lower service violations than alternative user-level methods of application-level resource monitoring and management. Given this observation, we feel ULS is preferred over SafeX as the first step toward supporting application-specific real-time services on commodity OSes. As evidence of the potential capabilities of ULS, we have implemented a user-space networking stack that avoids data copying via the kernel and allows packet processing to take place in the context of a kernel bottom half without explicit process scheduling, thereby increasing data throughput and reducing jitter. ULS requires no special hardware support other than page-based hardware protection and timer interrupt support to ensure predictable and low-latency execution of application-specific services.. Future work involves extending our ULS approach to multiprocessor platforms, and to provide safe and predictable resource management support for entire virtual machines rather than the more simplistic services and handlers currently supported. There have been a number of related research efforts that focus on OS structure, extensibility, safety, and service invocation. Extensible operating systems [Small and Seltzer 1996; Bershad et al. 1995; Chiueh et al. 1999; Jones 1993; Ghormley et al. 1997] aim to provide applications with greater control over the management of their resources. Additionally, microkernels [Accetta et al. 1986 ] and exokernels [Engler et al. 1995 ] offer a few basic abstractions, while moving the implementation of more complex services and policies into application-level components. By separating kernel-and user-level services, microkernels introduce significant amounts of interprocess communication overhead, although it has been argued that by leveraging hardware support many such costs can be made to disappear [Liedtke 1995] . In effect, our sandboxing technique provides a way to construct microkernel services without the inherent costs of heavyweight interaddress space communication. We support this without the need for esoteric hardware features (e.g., segmentation as used by Palladium [Chiueh et al. 1999] ) to implement fine-grained logical protection domains, so that the latency to invoke application-specific services is limited to the cost of an upcall and the flush of a TLB entry. Consequently, unbounded delays due to, for example, priority-based scheduling of process address spaces do not affect the timely execution of system service extensions. It is worth noting that several other research groups have, in contrast to our work, leveraged Linux to build real-time and QoS-based systems, such as RTLinux [Yodaiken and Barabanov 1997] , RED-Linux [Wang and Lin 1999] , and QLinux [Sundaram et al. 2000] . RTLinux is a small, hard, real-time kernel that executes Linux as a separate thread. RED-Linux provides a general scheduling framework to support different realtime scheduling policies within the kernel, while QLinux provides QoS guarantees to multimedia applications using special thread, packet, and disk scheduling policies. However, none of these systems has focused on combined hardware and software support to separate application-specific services from the core kernel, and none has focused on efficient and predictable methods by which services can be invoked so that scheduling and context-switching overheads are largely eliminated.. This leads to another area of related research, which focuses on service invocation, kernel event notifications [Banga et al. 1999; Lemon 2001] and upcalls [Clark 1985; Gopalakrishnan and Parulkar 1998 ]. Much of this work is concerned with the way to trigger user-level services or handlers due to some condition or event in the kernel. With our ULS approach, we enable upcalls to be triggered no matter which address space is active at the time of a kernel event, thereby greatly reducing the overheads of service invocation. Almost all other approaches still involve the scheduling of process address spaces in which to handle events from the kernel.. Finally, while others have considered methods to instrument applications, to intercept requests for resources such as CPU cycles, memory, and bandwidth [Chang et al. 2000] , the emphasis of our ULS and SafeX work is to develop safe and predictable execution domains in which application-specific services may be deployed. Our work enables COTS systems to be extended with resource management methods to improve and/or guarantee qualities of service [Rajkumar et al. 1998 ] to individual applications without the need for entire QoS architectures [Abdelzaher and Shin 1998; Rosu et al. 1998 ] to be constructed. As stated above, such execution domains do not suffer from scheduling and context-switching overheads as would be the case for services mapped into traditional process address spaces.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The type discipline of behavioral separation We introduce the concept of behavioral separation as a general principle for disciplining interference in higher-order imperative concurrent programs, and present a type-based approach that systematically develops the concept in the context of an ML-like language extended with concurrency and synchronization primitives. Behavioral separation builds on notions originally introduced for behavioral type systems and separation logics, but shifts the focus from the separation of static program state properties towards the separation of dynamic usage behaviors of runtime values. Behavioral separation types specify how values may be safely used by client code, and can enforce fine-grained interference control disciplines while preserving compositionality, information hiding, and flexibility. We illustrate how our type system, even if based on a small set of general primitives, is already able to tackle fairly challenging program idioms, involving aliasing at various types, concurrency with first-class threads, manipulation of linked data structures, behavioral borrowing, and invariant-based separation. The purpose of this work is to introduce and develop the concept of behavioral separation as a general principle for disciplining interference in higher-order imperative concurrent programs.. Statically verifying that higher-order imperative programs do not go wrong in the presence of possible interference has proven to be a challenging task, and a fertile ground for research since the seminal work of Reynolds [36] . In general, two program fragments interfere when the effects generated by one fragment may change the state visible to the other, typically due to aliasing or to concurrency. Some forms of interference are "bad", and may cause catastrophic failure, such as read/write races when accessing the same memory cell. Other forms of interference are "good" and even required, such as the interference between a producer and a consumer running concurrently, and sharing a thread-safe state-full queue, or Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. POPL'13, January 23-25, 2013 , Rome, Italy. Copyright c 2013 ACM 978-1-4503-1832-7/13/01. . . $15.00 the interference between the head and tail references of a linked list structure. "Interference is the essence of concurrency", as Cliff Jones frequently recalled [26] . An ongoing issue to overcome is then to find techniques for disciplining interference between different usages of the same objects and ensure safety, while being able to address increasingly sophisticated programming idioms.. Significant advances have been achieved recently towards these general goals, in particular by the separation logics of O'Hearn and Reynolds [33, 39] and by several substructural type and effect systems, e.g., [1, 3, 8] . Namely, separation logic supports expressive forms of local reasoning, based on the use of the separating conjunction in combination with fractional permissions [5, 9] to characterize the fine structure of program states. Extending such statebased techniques to tackle the sophisticated program idioms arising in modern higher-order imperative concurrent programming is thus both promising and challenging [40] .. In this work, we radically depart from a state-based view towards a behavioral view of program assertions, and introduce the notion of behavioral separation. Behavioral separation builds on concepts originating in behavioral type systems and separation logics, but shifts the focus from the separation of static program state properties towards the separation of dynamic usage behaviors of runtime values. More concretely, we develop a type structure that systematically explores the concept of behavioral separation to enforce safety of programs, ruling out "bad" interferences in the presence of aliasing and concurrency. Our presentation is grounded on a core ML-like programming language with concurrency primitives, designed as a convenient abstraction for a large family of languages supporting higher-order imperative concurrent programming.. Behavioral types [16, 21, 24] , based on process algebras, characterize the interface of a system not just as a specification of the static type of exchanged messages, but more importantly as a specification of its dynamic behavior. Likewise, our behavioral separation types specify how program values may be safely used by client code, but, unlike pure behavioral types, are able to capture fine-grained interference control disciplines. A key novelty of our approach consists in uniformly combining in the same type structure "temporal" operations, such as sequential separation, important to capture constraints on traces, with "spatial" operations, such as parallel separation and isolation, important to discipline aliasing and concurrency. Remarkably, we carry out our development in the context of a clean substructural type theory, in which all type operators satisfy natural algebraic properties, based on a λ-calculus extended with imperative references and concurrency constructs.. Behavioral separation types promote information hiding, compositionality, and flexibility, since type assertions talk about separation constraints on usage behaviors as externally perceived by the programs which use them, rather than about the internal structure of program state or code. As will be clear from our examples, behavioral separation types are also expressive and flexible enough for proving safety of programs combining features still challenging for (and even out of reach of) existing proof methods, including general higher-order store, aliasing / sharing at all types, linked data structures, borrowing of local behavior, first-class threads, and invariantbased separation, based on typed synchronization constructs. In this section, we motivate the general concept of behavioral separation, informally introducing our core programming language and the various behavioral separation type operators by going through a sequence of examples. Consider first the following implementation of a collection ADT, where we assume list elements to be natural numbers and the representation data structure to be a linked list.. let We define four operations on collections: the initializer init, which sets the collection identifier (a string); the getId operation, which returns the collection identifier; the add operation, which adds a new element to the collection; and the scan operation, which traverses the linked list, visiting each node in sequence. We model ADT "objects" by tuples of closures sharing memory locations, and "classes" by object generating functions, along standard lines. In our language, tuple fields are bound to expressions (quoted code), to be evaluated only after field selection (as e.g. in [38] ). The var x in e block creates a ML-style heap allocated variable, where the created cell can survive the lifetime of the body e, embedded in the value returned (cf. let x = ref(nil) in e in ML).. In the code shown above, the private hd variable refers to the head of the linked list, and is shared by the add and scan operations. We represent references to list elements by variant values, with options NULL(nil) abbreviated NULL and representing the null reference, and NODE(n), representing a list node. In NODE(n), n is a tuple with fields setElt , getElt, setNext , and getNext . Notice that the latter two fields access the heap variable next , which references the next node, if any, and is local to the given node.. Using standard functional and product types we could assign to newColl a type as 0→SC , where 0 is a "unit" type, and SC a record type representing the collection "objects". Such a record type would essentially specify a flat interface, listing the signature of the operations available, each one modeled as a typed field. Quite differently, in our system types specify value usage behaviors, rather than value structure. As a first example consider SC init:str| →0 ;(getId :str add :nat| →0 scan:0) * The type SC specifies a (possible) usage behavior offered by a collection. Intuitively, the type SC says that collections may be used by first calling the init "method", and then the getId , add , and scan "methods", in iterated choice. First, it offers a label selection usage, denoted by the label selection type init:str| →0. The usage consists in selecting the init label to get a value of type str | → 0. The stop type 0 specifies that no usage is available.. The operator (−)| →(−) is our primitive functional type. U | → V is a type for functions which do not interfere unsafely with their argument, and specifies a single usage of a value as a function, at the appropriate argument U and return type V . The U | →V does not correspond to a standard function type U →V , which can nevertheless be encoded in our system as will become clear later. It does neither correspond to the linear arrow, nor to the arrows of separation [33, 39] or bunched logic [32] , even if it is closely connected to all of these. In a state-full, concurrent programming world, there are just too many ways of using a function object. In conjugation with other type operators, the functional type U | →V allows much of such variety to be modularly approached.. Sequencing of behaviors is expressed by the sequential separation type operator (−) ;(−). A value typed by U ; V first offers to clients the usage behavior U and only after V . In our example, after the init:str| →0 usage behavior, the collection value offers a usage (getId :str add :nat| →0 scan:0). * . This last type specifies the iterated choice between the selection of fields getId , add , and scan, each one yielding a value of respectively type str, nat | → 0, and 0. Choice between alternative behaviors is expressed using intersection types. A value typed by U V offers to clients the choice between behaviors U and V , since it can provide both U and V , alternatively. The star type U * denotes the iteration of U . Clearly, the type operators just described may express rich sequential protocols. Still, they are not expressive enough to express aliased or concurrent usages, due to the strict linearity they enforce. A more flexible and still safe usage type for our collection would allow, after initialization, the getId operation to be always available, concurrently with a add or a scan operation. To express this possibility we use the parallel separation type operator (−) |(−).. In general, a U | V type asserts that behaviors U and V may be safely used by causally independent clients, either due to aliasing or to concurrency, without incurring in unsafe interference. Such a parallel usage only completes when both behaviors U and V complete. Exploring parallel separation, we may assign to function newColl the more flexible type 0| →CC where CC (init:str| →0) ;(!getId :str |(!scan:0 ; add:nat| →0). * ). The type CC asserts that, after initialization, a collection provides two independently usable behaviors, one of type !getId :str and other of type (!scan:0 ; add:nat| →0) * , composed using the parallel separation type operator (−) |(−).. The type operator !(−), used in !getId :str and !scan:0, specifies an unbounded number, possibly zero, of separated parallel usages (parallel in the sense of (−) |(−)). In particular, the type !getId :str allows an unbounded number of (possibly concurrent) aliases to access the field getId :str. Then, the type (!scan:0 ; add:nat| →0) * specifies a usage consisting of the interleaved repetition of some parallel usages of the scan:0 behavior followed by the add:nat| →0 behavior. Only after all the scan operations selected in the !scan:0 phase conclude, will add:nat| →0 become again available. Since there is no obligation to pick any scan:0 operation in the !scan:0 phase, the type CC also allows any number of add:nat| →0 operations to be sequentially performed.. Notice that the behavior of a newly created collection c is completely separated, or isolated, from context: no behavioral dependencies exist between c:CC and the behavior of other values in a program using the collection. We express isolation, which plays an important role in our framework, by a specific type operator •(−). We thus assign to the function newColl the type 0 | → •CC .. Let us now consider some code snippets using the collection type just defined, and discuss valid (and invalid) typings. Both code fragments are validated by our type system. In the first one, it is clear that the usage of c follows the intended type. In the second one, the intended usage type of c is also not violated, even if behaviors that appear parallel separated in the type (e.g., c:getId :str and c:scan:0) are sequentially used in the code. Clearly, if a value may be safely used according to U | V , it may also be safely used according to U ; V . Subsumption principles as this one are captured by subtyping, a pre-order on types written U <: V . In particular, subtyping satisfies the exchange law [19] . of which U | V <: U ; V is a special case.. The next examples illustrate behavioral "borrowing", where fragments of the behavior of c are temporarily used by some function, before being given back to the caller context.. In the second case the borrowing function is used twice, at different places of the global behavior. The borrowed type is declared in the function domain, e.g., f :(init:str | → 0) | → 0. On the other hand. attempts to use add before init, and is rejected by our type system. More interesting examples illustrate borrowing of behavior through the store, which our type system is able to handle in a natural way, even in a higher-order setting. Consider the following code snippet. It respects the expected behavioral constraints on the value c and the heap allocated variable a, even if the behavior of c is temporarily accessible at a, and is in fact typeable in our system. Heap variables are assigned behavioral separation types expressed in terms of use use, read rd(U ) and write wr(U ) capabilities, and related by subtyping axioms. We show some of them here var <: use ; var use <:. The first two axioms say that a single use of a variable consists in writing on it a value of type U , followed by a matching read phase. The following axioms specify how the reading phase may be behaviorally separated, depending on the type of the stored value. The next example illustrates higher-order borrowing through the store, the function attached to the add field is itself stored in memory, before being called.. var a in (a := c.add ; (a 1); c.scan )). This code does not violate any constraints imposed by the type of c, even if a collection "method" (a functional value) is extracted by selecting c.add and stored in the temporary heap variable a. These last two examples get past our typing discipline, because the type system keeps track of global separation constraints between all the values in the scope, and relies on sequential and parallel frame reasoning to locally replace behaviors in behavioral separation type assertions. In the last example, the function of type nat | → 0 is required to be used (exactly once) before scan is selected in c, even if its behavior is temporarily stored in the variable a, respecting the initial footprint of c.add of in the global behavioral separation type. The type pre-condition of a := c.add is a:use |(c:add :nat | → 0 ; c:scan:0 ; · · · ), and the post-condition is a:rd(add :nat | → 0) ; c:scan:0 ; · · · . Notice that this last type sequentially constrains the behaviors of a and c, forcing a to be read before using c. The ability to specify global separation constraints, involving several values, seems essential for the expressiveness of our system. As a further illustration of this point, consider. Here, the init qualifier is selected, but since the associated functional behavior (bound to m) is not actually exercised, the initialization of the local hd variable is not performed, causing the scan operation to "crash" in the case expression. Of course, this code does not type check under our current assumptions, since it does not preserve the frame conditions imposed by the intended behavioral separation protocols. Indeed, c.scan would need to be typed under the pre-condition (m:str | → 0) ; c:scan:0 ; · · · , which states that m must be used (as a function) before progressing with the continuation behavior of collection c, which is not possible.. The type system systematically uses local reasoning and frame principles on behavioral separation assertions to modularly compute effects of program fragments. As another example, consider . Isolated types offer a safe escape from the strict locality discipline, allowing isolated behaviors to be fully (and soundly) separated from a global type in which they might appear embedded. As a further illustration, the following code is typeable by assigning •CC | → 0 to f so that it captures the argument full behavior (storing it in heap variable a) rather than borrowing it.. We now consider some examples with concurrency. The parallel expression (e1 e2) clearly brings up the possibility of interference. The next couple of examples are safe, and type-check in our system. On the other hand, the code snippets. violate the intended behavioral separation constraints, and are rejected by the type system. In the last case, although the function f may be given type (add:nat | → 0) | → 0, the application (f c) is not typeable, since the type CC cannot provide footprints for separately typing function and argument (no parallel separated add capabilities are available on c). On the other hand, the following similar looking code is safe and well-typed.. The form (e1 e2) is actually derived from primitive fork and wait thread-based concurrency constructs. Thread references are firstclass values in our language, created by the fork(e) expression. The interesting operation on threads is to wait for their return value. Let is not well-typed under the same assumptions: it breaks the separation constraints required by the type of c. Such type requires c.scan and (c.add 1) to be sequentially separated, but overlapping may occur at runtime, causing unsafe interference.. As noticed before, it is not sound to assign to our collection a type allowing the add operation to be used concurrently with scan operations. That would violate the intended usage protocol of the internal state, causing a write/read race on heap variable hd. However, our language allows "critical regions" in the code to be sequentialized, and eventually typed by invariant-based separation. Invariant-based separation allows isolated behaviors to be repeatedly interleaved in the global behavior, as far as the associated invariant conditions, expressed by a conveniently chosen type assertion, are preserved. In our example, this could be achieved, e.g., by adding a new local heap variable inv to the collection, and wrapping the uses of hd in add and scan as follows. To type this code, we associate an assertion hd :rd(!•PNode) ; var to the heap variable inv , which expresses an invariant condition protecting its footprint. Our type system is then able to assign to the concurrent collection the following type, which allows, after initialization, operations getId , add and scan to be unboundedly aliased, or shared by several active threads.. Of course, the main novelty to highlight here is not the familiar reasoning technique for lock invariants, but the way our type discipline elegantly captures it. Even if based on a few fairly general principles, it can be effectively used to reason about safety properties of higher-order concurrent programs involving difficult to handle scenarios of aliasing and concurrency. We are not aware of related proposals, able to address the same set of (realistic) programming idioms, and based on a a similarly general foundation, as we have achieved here. This paper makes the following contributions:. • We motivate and introduce the concept of behavioral separation as a general principle for disciplining interference in higherorder imperative concurrent programs.. • We present a behavioral separation type system for a λ-calculus with imperative and concurrency constructs. We show soundness of the system, proving type preservation under reduction (Theorems 4.3 and 4.5) and progress (Theorem 4.4).. • We illustrate, by means of many examples, how our type system, even if based on a small set of very general primitives, is already able to tackle fairly challenging program idioms. We have introduced behavioral separation as a general principle for disciplining interference, due either to aliasing or concurrency, by combining concepts from separation logic and behavioral type systems. We have designed a behavioral separation type system that illustrates the concept using a higher-order imperative functional language extended with concurrency and synchronization primitives. Our type system is proven sound using proof theoretic techniques.. We have also shown that the expressiveness of our approach goes beyond the state of the art for type-based verification of aliasing and concurrency, and provided several challenging examples involving fine-grained state manipulation, thread based concurrency, and synchronization constructs. Further examples, including the implementation of a concurrent queue based on a double linked list can be found in [13] . In ongoing work, we have already designed an algorithm that can effectively type check programs in our core language with a reasonable annotation burden -these results will be reported elsewhere. We are also investigating generalizations to invariant-based separation, along the lines suggested in Section 4.5, and the extension of our basic framework with dependent types, which would support more precise specifications. The concept of separation results from a research stream whose origins can be traced back to the seminal works of Reynolds on syntactic control of interference [36, 37] . Separation logics extend classical Hoare logic with new connectives, in particular the separation conjunction, which allows to specify the fine-grained structure of states in programs manipulating references, and enables local reasoning to successfully tackle programs with references [39] and concurrency [33] . More recently, separation logic has motivated the introduction of Hoare Type Theory [31] , and has been extended to languages with higher-order store [40] . These works focus on the identification of higher-order frame principles for state-based local reasoning. The idea of assigning a parallel separation type to some value, even when there is (safe) interference between the implementations of the separated behaviors, is reminiscent of concepts explored in fictional separation [25] , concurrent abstract predicates [17] , and superficially substructural types [28] . Again, these works are carried out within state-based separation logics. We have shown how a type based approach to behavioral separation, in principle less precise for specification purposes than a full logic, can already be used to enforce interference safety on programming languages idioms and features not yet in the scope of such approaches, such as higher-order store and first class concurrency. Work on spatial logics for concurrent systems [11, 15] offered remote inspiration for our approach, but explored different notions of "separation", useful to reasoning about distribution and mobility, rather than about interference. Although the fundamental notion of separation applies to many kinds of computational structures [14, 20] , the idea of combining separation with behavioral types to discipline interference in a realistic programming language, as we do here, does not seem to have been considered before. Various forms of behavioral types have been independently introduced by several authors with the intent of classifying usage patterns of computational objects [16, 21, 24] . Some of these works have motivated more refined verification techniques, for example, to check resource usage disciplines [23] in functional programs. A particular case of behavioral types are the so-called session types, intended to discipline message exchanges between partners in distributed systems. Although initially proposed for systems with interaction between exactly two partners [21] , session types have been extended to systems with an arbitrary number of participants [22] . Our notion of type assertion is loosely related with the notion of global type introduced in [22] , in the sense that it needs to talk about the joint behavior of several entities. A version of session types to discipline interactions between concurrent objects was developed in [18] , but does not attempt to deal with interference or aliasing. More recently, the first author and Pfenning developed an interpretation of session types in linear logic [12] , which also inspired some aspects of the theory presented here. Connections between session types and separation logic have also been investigated in [41] , but focusing on disciplining the transference of resources in process communications. In prior work, we attempted a very preliminary approach to the concept of behavioral separation [10] . However, the developments in this paper clarify the notion of behavioral separation type in the context of a clean substructural type theory, based on a λ-calculus with imperative and concurrency constructs, and are much more general and expressive.. Several works have proposed type-based approaches to discipline aliasing and concurrency control in various programming languages, usually exploiting type and effect systems [1, 8] . Ownership types have also been studied to discipline aliasing and concurrency [6, 7] . Some of these works have led to the development of powerful programming tools [27] . Typically, these works do not focus on capturing the dynamic behavior of resources at a deeper level, as we do here, but on tracking occurrences of identifiers, locks, permissions, regions, and data dependencies, essentially resorting to linearity. An important exception is typestate, which uses a state-based approach to specify resource usage protocols in object oriented languages [3] . A key ingredient of the typestate approach is the use of primitive permissions to capture usage idioms, rather than resource behavior. In parallel research we are investigating combinations of separation with typestate [29] . Techniques to support expressive borrowing idioms in the context of typestate have been recently proposed in [30] .

O O O O O B O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O B O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O B O B I O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O


Exploitation and exploration in a performance based contextual advertising system The dynamic marketplace in online advertising calls for ranking systems that are optimized to consistently promote and capitalize better performing ads. The streaming nature of online data inevitably makes an advertising system choose between maximizing its expected revenue according to its current knowledge in short term (exploitation) and trying to learn more about the unknown to improve its knowledge (exploration), since the latter might increase its revenue in the future. The exploitation and exploration (EE) tradeoff has been extensively studied in the reinforcement learning community, however, not been paid much attention in online advertising until recently. In this paper, we develop two novel EE strategies for online advertising. Specifically, our methods can adaptively balance the two aspects of EE by automatically learning the optimal tradeoff and incorporating confidence metrics of historical performance. Within a deliberately designed offline simulation framework we apply our algorithms to an industry leading performance based contextual advertising system and conduct extensive evaluations with real online event log data. The experimental results and detailed analysis reveal several important findings of the EE behaviors in online advertising and demonstrate that our algorithms perform superiorly in terms of ad reach and click-through-rate (CTR). In the advertising industry, an ongoing shift in spending from traditional media to the Internet has been widely observed during the past decade. Since 2006, the year-overyear growth of Internet advertising spending has eclipsed other conventional media such as newspaper. As marketers realize that online spending is more effective at influencing consumers and delivering more measurable results, nowadays online advertising has become a major business. Based on a recent study by Interactive Advertising Bureau, online advertising revenue in the first half of 2009 reached over 10.9 billion US dollars [4] .. A significant part of this market consists of ads delivery distributed through three main advertising channels: display ads, sponsored search and contextual advertising. In this paper, we focus on contextual advertising that refers to the placement of ads within the content of a generic Web page for a user, but the ideas illustrated here are generally applicable to other distribution channels as well. In contextual advertising, usually there is a commercial intermediary, called ad-network, in charge of optimizing the ad selection with a three-fold goal of increasing revenue (shared between publishers and ad-network), improving return-oninvestment of advertisers, and enhancing user experience.. One of the most prevalent pricing models for contextual advertising is that advertisers pay a certain amount for every click on an ad (cost-per-click or CPC). There are also other models: cost-per-impression where advertisers pay for the number of exposures of an ad, and cost-per-action where advertisers pay only if an ad leads to a sale or a similar transaction. In the CPC model, the revenue of an ad-network is crucially dependent on the click-through-rate (CTR) that can be viewed as the probability that an end-user would click on an ad when shown on a Web page.. From the perspective of ad-networks, it is always preferable to show ads with high expected revenue, which depends on both ad CTR and its bid in the CPC model. Unfortunately, ad CTRs cannot be known a priori, and thus many ad-networks' ranking systems use historical CTRs instead. This approach is usually referred to as performance based or click feedback model. However, this kind of systems are likely to suffer from the Matthew effect (the rich get richer and the poor get poorer) in the long run. The ads with large numbers of impressions and high CTRs are usually the top candidates for displaying. Therefore they will get even more impressions (and sometimes clicks) in the future. On the other hand, the ads with low impressions and potentially high CTRs might unluckily have few or no clicks, and thus are considered inferior performers due to the bogus historical CTRs associated with the sparse data. Their chances for future exposure will be even slimmer regardless of their true CTRs. This is especially bad news for a fast changing marketplace where we constantly have new ads joining every hour. In an exploitation only system, these newcomers are not as competent as the established good performers, and it would be difficult for them to catch up. In the long term, most impressions will go to a small set of ads. In general, this trend is not desirable as it could cause serious problems:. 1. Keeping showing the same ads repeatedly to the same users easily leads to user fatigue. That is widely considered as very unpleasant user experience and could result in a drop in clicks [7] .. 2. Either over-exposing ads of the same advertisers in a short period or under-exposing in a long period can severely hurt economic efficiency for both advertisers and ad-network operators. Some advertisers will quickly burn out their budgets while others cannot spend their planned ad dollars due to the lack of exposure.. In order to address these issues, an ideal way is to exploit those established good ads for short-term capitalization, and at the same time explore the unknown (or less known) ad space to discover potentially better ads for the long term. This is a typical tradeoff between exploitation and exploration (EE), frequently studied with the multi-armed bandit problem in reinforcement learning [12, 13] . More specifically, our dilemma can be viewed as a variation of the multi-armed bandit problem, defined as iteratively choosing one out of multiple actions and receiving a reward drawn from a distribution associated with this action. The goal is to find the optimal action sequence that maximizes the accumulated rewards. In the scenario of contextual advertising, we select a small number of ads for each page view and receive a user feedback (click or non-click) as reward.. One classical solution to the multi-armed bandit problem is the -greedy strategy [12] . With probability 1-, this algorithm chooses the best action based on current knowledge; and with probability , it chooses any other action uniformly. The parameter essentially controls the tradeoff between exploitation and exploration. One disadvantage of this algorithm is that the optimal value is difficult to decide in advance. Moreover, we may need to dynamically adjust the EE tradeoff at different stages of the experiment. This concern can be addressed by a slightly different strategy called -decreasing [12] , where the value of decreases as the experiment processes. In other words, this algorithm focuses more on exploration at the beginning and gradually shifts to exploitation at the end.. Compared to the standard multi-armed bandit problem with a fixed set of possible actions, we face a fast changing marketplace in contextual advertising. Old ads may expire and new ads may emerge everyday. Therefore it may not be desirable to monotonically decrease the effort on exploration as the -decreasing strategy does. Instead, we want to adaptively balance the tradeoff between exploitation and exploration in response to the data change. In this paper we propose two novel approaches that achieve this goal.. In the first method, we extend the -greedy strategy by updating the value of dynamically (not necessarily decreasing). At each iteration, we run a sampling procedure to select a new from a finite set of candidates. The probabilities associated with the candidates are initialized uniformly and updated with the Exponentiated Gradient (EG) algorithm [9] . As shown in Section 4, this updating rule increases the probability of being chosen for a candidate if it leads to a user click.. In the second method, we improve a different aspect of the -greedy algorithm, namely the random exploration strategy. As we mention before, the marketplace in contextual advertising can change rapidly, and different ads have very different histories. Some old ads have already been shown many times while new ones have few or no impressions. As a result, it is not desirable to explore ads in a random way because it will waste opportunities on the established bad performers. Instead, we only want to explore the ads that might lead to higher revenue in the long run, i.e., ads with bogus poor performance due to sparse data or even no history. To address this, we introduce an impression based confidence metric and use it to decide which ads need to be explored. This confidence metric also serves as a dynamic switch between exploitation and exploration. When the confidence increases to a certain level, some exploration opportunities will be automatically shifted to exploitation.. In this paper, we set up an offline simulation framework from real event logs in an industry leading performance based contextual advertising system, and evaluate the shortterm and long-term impacts of EE on the system performance. The experiment results demonstrate that our proposed methods can greatly improve the ad reach and empirical CTR. Compared to an exploitation only baseline, both EE algorithms increase the ad reach by more than 25%. While the dynamic -greedy algorithm with Exponentiated Gradient converges to a slightly lower CTR than its best fixvalue counterpart, its fast convergence rate makes it more competent in a real system. Furthermore, the confidencebased algorithm improves CTR almost by half over the baseline and about 20% over other EE algorithms. In this paper, we study the problem of exploitation and exploration in online advertising and propose two novel approaches that adaptively balance the tradeoff between them. In order to evaluate the performance of the proposed algorithms more accurately, we introduce an offline simulation framework that provides an isolated environment as well as mimicking the online system and user behaviors. Under this framework, we compare our algorithms with a pure exploitation baseline and two other standard EE strategies, using an industry leading performance based contextual advertising system with real online event data. The experimental results demonstrate that our algorithms, especially the confidencebased approach, perform superiorly on the measurement of ad reach and CTR in various configurations. In the future, we are going to experiment with more sophisticated confidence measurements such as the variance of posterior CTR distribution using Beta-Binomial or Gamma-Poisson models. Another direction is to combine the advantages of the confidence-based algorithm and a dynamically learned parameter using the Exponentiated Gradient method. The exploitation and exploration tradeoff was first formally studied in reinforcement learning in 1980's, and later flourished in other fields of machine learning [12, 13] . Very frequently used in reinforcement learning to study the EE tradeoff, the multi-armed bandit problem (also sometimes called K-armed bandit) was originally described by Robbins [11] . Agarwal et al. recently applied this technique to maximize total clicks on a Web content module [1]. Langford et al. extended this problem to the case where there is some side information that helps to make a better decision [10] . They called the new setting "contextual bandit problem", introduced the epoch-greedy algorithm and derived its regret bound.. The confidence-based approach proposed in this paper is partially inspired by the UCB algorithm [2] that ranks the actions based on the combination of estimated rewards and their confidence intervals. However, given the large number of ads to be selected, the number of impressions received by each ad is usually small, leading to an inaccurate estimation of confidence intervals and consequently the limited performance of the standard UCB algorithm. Thus, unlike the UCB algorithm that is a deterministic strategy in choosing actions, the proposed confidence-based EE algorithm is a hybrid approach that combines UCB with the -greedy algorithm. By introducing randomness into the UCB algorithm, we alleviate the trouble in estimating confidence intervals and achieve more reliable performance.. Another closely related area to the discussion in this paper is the click feedback system in online advertising. Click feedback has a relatively long history in the nascent online advertising industry. As early as in 2002, Google Inc. incorporated click feedback into its sponsored search systemAdWords [3] . In more recent years, similar ideas are adopted into other distribution channels of online advertising as well. For example, Chakrabarti et al. described a contextually advertising system that combines ad relevance with historical impression and click information through a logistic regression model [6] . The performance-based contextual advertising system in this paper roughly shares the same idea. In this section, we describe our experiment design to evaluate Algorithms 2 and 3 and present the results. We are interested in evaluating the long-term effects of EE algorithms, but it takes time to explore the unknown space and discover new ads with good performance. During such a long time period, CTR may change significantly even if the system remains the same. Therefore we cannot simply evaluate an EE algorithm by comparing the CTRs of the system before and after applying it. Due to the restrictions of setting up an isolated environment in the online system, we. A ={a1, a2, · · · , an}: an ad list ranked by CFB scores {x1, x2, · · · , xn}: the impressions of the above ads r: number of reserved ads q: number of ads in the promotional queue : probability to select ads in the promotional queue w: impression upper-bound for exploration 2: Output:. F : A list of ads re-ranked by the EE algorithm 3:. end if 20: until P or A is empty 21: F ⇐ F ∪ P ∪ A instead introduce an offline simulation framework for more accurate evaluation. This system is able to mimic emitting the online events, running the CFB model, as well as setting up controlled experiment buckets to perform apple-to-apple comparisons between a pure exploitation baseline and various EE algorithms.

B I I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I B I O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Polymorphic regular tree types and patterns We propose a type system based on regular tree grammars, where algebraic datatypes are interpreted in a structural way. Thus, the same constructors can be reused for different types and a flexible subtyping relation can be defined between types, corresponding to the inclusion of their semantics. For instance, one can define a type for lists and a subtype of this type corresponding to lists of even length. Patterns are simply types annotated with binders. This provides a generalization of algebraic patterns with the ability of matching arbitrarily deep in a value. Our main contribution, compared to languages such as XDuce and CDuce, is that we are able to deal with both polymorphism and function types. We investigate an ML-style type system, with variants, function types and prenex polymorphism, but trade type inference for a powerful subtyping relation. Our main motivation for this work is the design of an extension of XDuce [7] with function types and polymorphism, which are crucial for programming in the large.. The type system is based on regular tree grammars. Types are similar to the algebraic datatypes of functional languages. For instance, we can define a list Cons (1, Cons (2, Nil)). A type for this value can be specified using the following definition of a type list with two variants. It would be interesting to extend our calculus with bounded quantification. An easy way is to provide a bound when abstracting over a type variable and check the bound at type application. The subtyping relation can be extended so that a type variable is a subtype of its bound. But this does not yield a complete subtyping relation: we encounter the same difficulties as Hosoya, Frisch and Castagna [6] . Our analyses should probably also be refined to take into account the bounds. We have not yet studied how they should be modified. The type system we present here is very close to the XDuce type system [7, 8] but also features polymorphism and function types. This is not immediately apparent as XDuce uses syntactic sugar for values and types so that its values look like fragments of XML documents.. CDuce [3] has a very powerful type system, which is basically an extension of XDuce type system with function types and intersection types. Pattern matching on functions involves subtype checks at runtime. We wanted to keep our type system as simple as possible, so we did not considered intersection types.. Hosoya, Frisch and Castagna propose an extension of XDuce type system with polymorphism [6] . Though there is no explicit type intersection construction, their types are closed under intersection. This is an elegant feature and it makes it possible to encode bounded quantification. But it interacts badly with polymorphism. In particular, the straightforward semantic interpretation of subtyping used in XDuce and CDuce yields a subtyping relation which is difficult to check. Hence, they propose a different interpretation based on marking values with type variables. They do not propose anything similar to our untyped semantics: their pattern matching construction is allowed to break abstraction and type variables are not allowed in patterns.

O O B B O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O B O O O B B O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O B O O B O O O O B I O O O B O O B O O O O O O O B B O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B I O O O O B O B O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O B O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O B I O O B O O B B B O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O B I O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O B O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O


Large-scale seismic signal analysis with Hadoop In seismology, waveform cross correlation has been used for years to produce high-precision hypocenter locations and for sensitive detectors. Because correlated seismograms generally are found only at small hypocenter separation distances, correlation detectors have historically been reserved for spotlight purposes. However, many regions have been found to produce large numbers of correlated seismograms, and there is growing interest in building next-generation pipelines that employ correlation as a core part of their operation. In an effort to better understand the distribution and behavior of correlated seismic events, we have cross correlated a global dataset consisting of over 300 million seismograms. This was done using a conventional distributed cluster, and required 42 days. In anticipation of processing much larger datasets, we have re-architected the system to run as a series of MapReduce jobs on a Hadoop cluster. In doing so we achieved a factor of 19 performance increase on a test dataset. We found that fundamental algorithmic transformations were required to achieve the maximum performance increase. Whereas in the original IO-bound implementation, we went to great lengths to minimize IO, in the Hadoop implementation where IO is cheap, we were able to greatly increase the parallelism of our algorithms by performing a tiered series of very fine-grained (highly parallelizable) transformations on the data. Each of these MapReduce jobs required reading and writing large amounts of data. But, because IO is very fast, and because the fine-grained computations could be handled extremely quickly by the mappers, the net was a large performance gain. It has long been recognized that in collections of seismic data recorded by the same instrument from sources in similar locations there will be many similar seismograms (e.g. Geller and Mueller, 1980, Poupinet et al., 1984) . Geller and Mueller (1980) attributed the similarity to the fact that for small magnitude earthquakes, propagation results in an effective low-pass-filtering of the signals to become essentially the Green's functions, so repeated ruptures of the same asperity produce the same seismogram.. Since the initial observations of doublets, researchers have exploited the phenomenon in a variety of different ways. Much work has centered on producing high-precision relocations of clustered seismicity by correlating the waveforms to obtain highprecision relative picks used to relocate the events. For example Fremont and Malone (1987) and Got et al. (1994) imaged structures underneath active volcanoes by relocation of multiplets. Rubin et al. (1999) imaged seismicity on creeping sections of the Hayward fault. Hauksson and Shearer (2005) relocated 327,000 Southern California earthquakes using waveform cross correlation.. Seismic tomography based on correlation of ambient seismic noise (Shapiro et al., 2005) , seismic coda (Campillo and Paul, 2003) , and higher-order methods e.g. C 3 (Stehly et al., 2006) has become a very important means of building high-resolution models of the Earth's crust and upper mantle. Although not the topic of this paper, these are data-intensive operations and could conceivably benefit from a Hadoop implementation at a sufficiently large scale of implementation. Waveform correlation can also be used as the basis for highly sensitive detectors. This application has been known since at least the 1960s (Anstey, 1966) and has been employed on numerous occasions since. Because the correlation "footprint" of high frequency signals is generally quite small, correlation detectors have been used almost exclusively as "spotlight" detectors aimed at small regions of interest. Harris and Dodge (2011) used correlation in combination with subspace detectors (e.g. Harris, 2006) in an automated system to track events in an aftershock sequence.. However, interest in using correlation to process events over broader regions has grown. For example Richards (2004, 2011 ) discovered that about 13% of 18,000 earthquakes recorded at regional distances in China were sufficiently well correlated that they could be detected and located using waveform correlation.. To make better use of correlation relationships in seismic data analysis we need to understand what fraction of observed seismicity displays correlation relationships as functions of observed properties (e.g. source-receiver distance, window-length, bandwidth), and correlated source differences (e.g. location, depth, magnitude). The longer-term goal is to understand the physics underlying observed correlation behavior in terms of source similarity and path effects.. However, an impediment to our ability to investigate seismogram correlation is the computational costs of determining these relationships for the tens of thousands of seismic events each year observed at each of the tens of thousands of stations where data is available. Without an extremely efficient way of performing these computations, it is simply too difficult and time consuming to perform the many correlation runs that may be required to achieve a deep understanding of the phenomena under study.. This paper presents a methodology to significantly improve the speed at which massive amounts of seismic event data can be processed to look for correlation behavior using a Hadoop architecture. A result is to explore and expose the correlation behavior of large collections of seismic data. An expected outcome after analysis of these results is completed will be to make correlation a more useful tool in both geophysical research and seismic event cataloging operational systems. The Hadoop model of distributing the data with the computations presents a paradigm shift for the data-intensive scientific computing community. It demonstrates the need to change algorithmic priorities to fully take advantage of these powerful systems. Instead of asking ourselves how we can decrease the IO burden, as we did in our original implementation of the waveform correlator, we now find ourselves asking how we can increase the parallelism of our algorithms. Whereas before we had lots of CPU which could not be fully utilized, now we have blazingly fast IO and imbalanced CPU load. The process of finding new ways to break apart one's algorithms into finer-grained sub-problems is at the heart of the Hadoop philosophy: scale out, not up. Based on our 1 TB test data set and the factor of 19 performance increase found by moving to the HADOOP architecture, we expect we will be able to re-correlate our entire $ 50 TB, $ 300 million waveform database in about 2 days instead of the original 42 days. This will dramatically improve our ability to conduct research on massive seismic datasets, and we intend to describe those results in future papers. The lessons of this study, making use of HADOOP to increase parallelism instead of reducing IO, can apply to many massive datasets of time series data, which are common in geophysics and other fields.. There is an important caveat, however: Namely the time cost of getting data into a Hadoop cluster. In our case the limitations are due to our NAS speed and to network throughput limitations. We estimate that one to two months may be required for an initial load of our complete holdings. For academic users who use data held at remote data centers, the latency would likely be much larger.. One solution may be to make the Hadoop cluster the only place holding waveforms and their metadata. In this model, users wishing to perform analytics submit jobs to the cluster, but users wishing to retrieve data make requests to a server which in turn retrieves the data from HDFS, packages it and returns it to the requester. We are studying this option for our own use.. The Hadoop ecosystem is evolving very rapidly with numerous SQL-over-Hadoop, streaming analytics, and time series databases under development. While this makes it hard to decide on an implementation right now, it seems very likely that in a year or two there will be some clear winners whose technology can be adopted for the seismological community.. Proof. In the NEAREST NEIGHBORS calculation, there is at all times a current event whose step number corresponds to the order it was removed from the queue in the breadth-first search. All of the current event's unseen neighbors are added to the queue together and assigned a step number greater than that of the current event. All previously unseen neighbors will be processed in a sequence together called the unseen sequence. □ By induction on the current event in the CORRELATE calculation, we will show that CORRELATE has correlated all of the current event's neighbors (the current candidates) by the time an incoming event is accumulated that is not a current candidate.. In the base case, at step 1, we accumulate the first incoming event from the bag of island events ordered by step number and promote it to be the current event. There are two possibilities: either the current event has an unseen sequence of current candidates to correlate, or it does not. If there is no remaining unseen sequence to process, then our claim is correct by definition. Thus we assume that the current event has an unseen sequence that needs to be processed. If the next incoming event is not a current candidate, then it must be part of another event's unseen sequence by definition of the connectivity of an island. By definition of the current event, the incoming event must be part of the unseen sequence of an event with a higher step number than the current event. But the current event's unseen sequence must necessarily come before the unseen sequence of any subsequent events by definition of the step number, and so this is a contradiction. Thus there are no more neighbors to correlate against the current event.. Suppose the claim holds for current events up to step k À 1. At step k, we remove the kth event accumulated from the queue and promote it to be the current event. We then proceed to correlate it against all the elements on the queue. If the current event had any neighbors with a lower step number than itself, they were correlated earlier in the process by our assumption. Thus the only current candidates remaining must come from incoming events. Without loss of generality, we can apply the same reasoning to incoming events as applied in the base case to demonstrate that all remaining neighbors must be part of the next incoming sequence, or there are none left to correlate against the current event, and so the CORRELATE algorithm does not miss any potential correlations.

O O O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O