A study of practical deduplication We collected file system content data from 857 desktop computers at Microsoft over a span of 4 weeks. We analyzed the data to determine the relative efficacy of data deduplication, particularly considering whole-file versus block-level elimination of redundancy. We found that whole-file deduplication achieves about three quarters of the space savings of the most aggressive block-level deduplication for storage of live file systems, and 87% of the savings for backup images. We also studied file fragmentation, finding that it is not prevalent, and updated prior file system metadata studies, finding that the distribution of file sizes continues to skew toward very large unstructured files. File systems often contain redundant copies of information: identical files or subfile regions, possibly stored on a single host, on a shared storage cluster, or backed-up to secondary storage. Deduplicating storage systems take advantage of this redundancy to reduce the underlying space needed to contain the file systems (or backup images thereof). Deduplication can work at either the subfile [Dubnicki et al. 2009; or whole-file [Bolosky et al. 2000] level. More fine-grained deduplication creates more opportunities for space savings, but necessarily reduces the sequential layout of some files, which may have significant performance impacts when hard disks are used for storage (and in some cases [Zhu et al. 2008] necessitates complicated techniques to improve performance). Alternatively, whole-file deduplication is simpler and eliminates file-fragmentation concerns, but at the cost of some otherwise reclaimable storage.. Because the disk technology trend is toward improved sequential bandwidth and reduced per-byte cost, with little or no improvement in random access speed, it's not clear that trading away sequentiality for space savings makes sense, at least in primary storage.. Microsoft as whole funded and enabled this research. Authors' addresses: D.T. Meyer, The University of British Columbia, Microsoft Research; email: dmeyer @cs.ubc.ca; W.J. Bolosky, Microsoft Research; email: bolosky@microsoft.com . Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested fromIn order to evaluate the tradeoff in space savings between whole-file and blockbased deduplication, we conducted a large-scale study of file system contents on desktop Windows machines at Microsoft. Our study consists of 857 file systems spanning 162 terabytes of disk over 4 weeks. It includes results from a broad cross-section of employees, including software developers, testers, management, sales and marketing, technical support, documentation writers, and legal staff. We find that while blockbased deduplication of our dataset can lower storage consumption to as little as 32% of its original requirements, nearly three quarters of the improvement observed could be captured through whole-file deduplication and sparseness. For four weeks of full backups, whole file deduplication (where a new backup image contains a reference to a duplicate file in an old backup) achieves 87% of the savings of block-based deduplication. We also explore the parameter space for deduplication systems, and quantify the relative benefits of sparse file support. Our study of file content is larger and more detailed than any previously published effort, which promises to inform the design of space-efficient storage systems.. In addition, we have conducted a study of metadata and data layout, as the last similar study [Agrawal et al. 2007 ] is now four years old. We find that the previously observed trend toward storage being consumed by files of increasing size continues unabated; half of all bytes are in files larger than 30MB (this figure was 2MB in 2000). Complicating matters, these files are in opaque unstructured formats with complicated access patterns. At the same time there are increasingly many small files in an increasingly complex file system tree.. Contrary to previous work [Seltzer and Smith 1997] , we find that file-level fragmentation is not widespread, presumably due to regularly scheduled background defragmenting in Windows [Scheduled Tasks 2010] and the finding that a large portion of files are rarely modified (see Section 4.4.2). For more than a decade, file system designers have been warned against measuring only fresh file system installations, since aged systems can have a significantly different performance profile [Seltzer and Smith 1997] . Our results show that this concern may no longer be relevant, at least to the extent that the aging produces file-level fragmentation. Ninety-six percent of files observed are entirely linear in the block address space. To our knowledge, this is the first large-scale study of disk fragmentation in the wild.. We describe in detail the novel analysis optimizations necessitated by the size of this data set. We studied file system data, metadata, and layout on nearly 1000 Windows file systems in a commercial environment. This new dataset contains metadata records of interest to file system designers, data content findings that will help create space efficiency techniques, and data layout information useful in the evaluation and optimization of storage systems.. We find that whole-file deduplication together with sparseness is a highly efficient means of lowering storage consumption, even in a backup scenario. It approaches the effectiveness of conventional deduplication at a much lower cost in performance and complexity. The environment we studied, despite being homogeneous, shows a large diversity in file systems and file sizes. These challenges, the increase in unstructured files, and an ever-deepening and more populated namespace pose significant challenge for future file system designs. However, at least one problem -that of file fragmentation, appears to be solved, provided that a machine has periods of inactivity in which defragmentation can be run. Studies of live deployed system behavior and usage have long been a key component of storage systems research. Workload studies [Vogels 1999 ] are helpful in determining what file systems do in a given slice of time, but provide little guidance as to the longterm contents of files or file systems. Prior file system content studies [Agrawal et al. 2007; Douceur and Bolosky 1999] have considered collections of machines similar to those observed here. The most recent such study uses 7-year old data, while data from the study before it is 11 years old, which we believe justifies the file system portion of this work. However, this research also captures relevant results that the previous work does not. Policroniades and Pratt [2004] studied duplication rates using various chunking strategies on a dataset about 0.1% of the size of ours, finding little whole-file duplication and a modest difference between fixed-block and content-based chunking. Kulkarni et al. [2004] found combining compression, eliminating duplicate identicalsized chunks, and delta-encoding across multiple datasets to be effective. Their corpus was about 8GB.. We are able to track file system fragmentation and data placement, which has not been analyzed recently [Seltzer and Smith 1997] or at large scale. We are also able to track several forms of deduplication, which is an important area of current research. Prior work has used very selective data sets, usually focusing either on frequent full backups Dorward and Quinlan 2002] , virtual machine images [Clements et al. 2009; Jin and Miller 2009 ], or simulation [Dubnicki et al. 2009 ]. In the former case, data not modified between backups can be trivially deduplicated, and in the latter, disk images start from a known identical storage, and diverge slowly over time. In terms of size, only the DataDomain [Zhu et al. 2008 ] study rivals ours. It is less than half the size presented here, and it was for a highly self-selective group. Thus, we not only consider a more general, but also a larger dataset than comparable studies. Moreover, we include a comparison to whole-file deduplication, which has been missing in much of the deduplication research to date. Whole file deduplication is an obvious alternative to block-based deduplication because it is lightweight and, as we have shown, nearly as effective at reclaiming space. Potential participants were selected randomly from Microsoft employees. Each was contacted with an offer to install a file system scanner on their work computer(s) in exchange for a chance to win a prize. The scanner ran autonomously during off-hours once per week from September 18 to October 16, 2009. We contacted 10,500 people in this manner to reach the target study size of about 1000 users. This represents a participation rate of roughly 10%, which is smaller than the rates of 22% in similar prior studies [Agrawal et al. 2007; Douceur and Bolosky 1999] . Anecdotally, many potential participants declined explicitly because the scanning process was quite invasive.

O B O O B O O O O O B O O O O O O O O O O O O O O O B O O O O O O B B O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Efficient inclusion for a class of XML types with interleaving and counting Inclusion between XML types is important but expensive, and is much more expensive when unordered types are considered. We prove here that inclusion for XML types with interleaving and counting can be decided in polynomial time in presence of two important restrictions: no element appears twice in the same content model, and Kleene star is only applied to disjunctions of single elements.Our approach is based on the transformation of each such type into a set of constraints that completely characterizes the type. We then provide a complete deduction system to verify whether the constraints of one type imply all the constraints of another one. XML schemas are an essential tool for the robustness of applications that involve XML data manipulation, transformation, integration, and, crucially, data exchange. To solve any static analysis problem that involves such types one must first be able to reason about their inclusion and equivalence.. XML schema languages are designed to describe ordered data, but they usually offer some (limited) support to deal with cases where the order among some elements is not constrained. These "unordered" mechanisms bring the language out of the well-understood realm of tree-grammars and tree-automata, and have been subject to little foundational study, with the important exception of a recent work by Gelade, Martens, and Neven [8] . Here, the authors study a wide range of schema languages, and show that the addition of interleaving and counting operators raises the complexity of inclusion checking from PSPACE (or EXPTIME, for Extended DTDs) to EXPSPACE. These are completeness results, hence this is really bad news. A previous result in [10] had already shown that the inclusion of Regular Expressions with interleaving alone is complete in EXPSPACE, hence showing that counting is not essential for the high cost. The paper [8] concludes with: "It would therefore be desirable to find robust subclasses for which the basic decision problems are in PTIME". Such subclasses could be used either to design a new schema language, or to design adaptive algorithms, that use the PTIME algorithm when possible, and resort to the full algorithm when needed. To this aim, it is important that (i) the subclass covers large classes of XML types used in practice, (ii) it is easy to verify whether a schema belongs to the subclass.. Our Contribution In this paper we define a class of XML types with interleaving and numerical constraints whose inclusion can be checked in polynomial time. These types are based on two restrictions that we impose on the Regular Expressions (REs) used to the define the element content models: each RE is conflict-free (or single occurrence) meaning that no symbol appears twice, and Kleene star is only applied to elements or to disjunctions of elements. These restrictions are severe, but, as shown in [5] and [6] , they are actually met by most of the schemas that are used in practice.. Our approach is based on the transformation of each type into an equivalent set of constraints. Consider, for instance, the following string type (a [ We will generalize this observation and give a technique to associate a complete set of constraints, in the six categories above, to any conflict-free type, and we will give a polynomial algorithm to verify whether, given T and T , the constraints of T imply those for T , so that T is included in T . (We will actually encode exclusion constraints as order constraints, hence dealing with five classes only.) We will formalize the constraints using a simple ad-hoc logic. We will describe the constraint implication algorithm by first giving a sound and complete constraint deduction system, and then giving an algorithm that exploits the deduction system. In particular, we will prove that the deduction of the constraints in one of the five categories does not need to consider the constraints in the other categories (apart from upper-bounds).. The ability to transform a type into a complete set of constraints expressed in a limited variable-free logic is used here to design an efficient inclusion algorithm. We believe that it can also be exploited for many related tasks, such as PTIME membership checking (which is N P -complete for REs with interleaving), type equivalence, and path containment under a DTD. Quite surprisingly, type intersection, which is usually simpler than type inclusion, turns out in this case to be NP-hard; the constraint-based approach was important in our discovery of the proof that we present here.. Paper Outline The paper is structured as follows. Section 2 describes the data model, the type language, and the constraint language we are using. Section 3 shows how types can be characterized in terms of constraints, and how inclusion can be encoded in terms of constraint implication. Section 4 describes a deduction system for type constraints. Section 5, then, illustrates a polynomial time algorithm for deciding type inclusion based on the deduction system of Section 4. In Section 6 we show that intersection is NP-hard. In Sections 7 and 8, finally, we briefly revise some related works and draw our conclusions. Inclusion for REs with interleaving, counting, or both, is EXPSPACE-complete, even if we consider the restricted subclass of CHAREs (with counting) [10, 8] . This result easily extends to XML types featuring these operators. We have introduced here a restricted class of REs with interleaving and counting. Our restriction is severe, but it seems to match reasonably well the measured features of actual DTDs and XSDs found on the web, and is extremely easy to define and verify. For this class of REs, we have proved that inclusion is in PTIME, a complexity that is surprising low, and trivially extends to DTDs and XSDs that use REs of this class for their content models. We have shown how to use classical algorithms to get a O(n 3 ) upper bound, but we feel that this could be easy lowered. We also proved that intersection has not the same complexity as inclusion (unless P=NP) but is, quite surprisingly, NP-hard.. Our result is based on the transformation of our REs into sets of constraints which completely characterize the expressions and are easy to manipulate. We believe that this constraint-based approach could be fruitfully used for other analysis tasks, such as, for example, type normalization, path minimization under a DTD, or a polynomial membership algorithm. The properties of unordered XML types have been studied in several recent papers. In [7] , the authors discuss the techniques and heuristics they used in implementing a type-checker, based on sheaves automata with Presburger arithmetic, for unordered XML types. The type language is an extension of the language we are considering here, and shares a similar restriction on the use of repetition types. The main purpose of the paper is to address scalability problems that naturally arise when working on XML types; as a consequence, they describe effective heuristics that improve scalability, but do not affect computational complexity.. Restrictions to RE languages that are similar to ours have been proposed many times. For example, conflict-free REs appear as "conflict-free DTDs" in the context of well-typed XML updates in [2] , as "duplicate-free DTDs" in the context of path inclusion in [12] , and as "single occurrence REs" in the context of DTD inference in [6] . The same restriction that we pose on Kleenestar can be found, for example, in [7] . Chain Regular Expressions (CHARE's) [6, 8] are also strictly related. They are defined as concatenations of factors, where each factor has a shape (a 1 + . . . + a n ), (a 1 + . . . + a n )?, (a 1 + . . . + a n ) * or (a 1 + . . . + a n ). + . As we discussed in Section 2.1, the first three classes of factors can be easily expressed in our language, using counting and interleaving. Factors like (a 1 + . . . + a n ). + cannot be expressed in our languages, but we could add them as a third class of base types {a 1 , . . . , a n }[ . * ]) = false. We did not consider these base types just for minimality. Simple expressions [5] have a more general syntax than CHAREs but the same expressive power, hence can still be managed through our approach.. We have cited many times paper [8] , where the complexity of type inclusion is studied for many different dialects of REs with interleaving and/or counting, showing that inclusion complexity is almost invariably EXPSPACEcomplete. In particular, this is shown to hold for chain-RE with counting, which are concatenations of CHARE factors, as defined above, and counting factors (a 1 + . . . + a k )[m..n] (with n = * and m ≥ 0), with no interleaving operator. In a sense, this hints that the conflict-free restriction, rather than the Kleene-star restriction, is crucial for our PTIME result. In the same paper, the authors introduce a sublanguage of CHAREs with PTIME inclusion, but that fragment is quite trivial, since it only includes counting factors (a 1 + . . . + a k )[m..n], with the further restriction that m > 0 and n = * , hence cannot express neither optionality nor unbounded repetition (neither * nor + ).

O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Identifying exogenous drivers and evolutionary stages in FLOSS projects The success of a Free/Libre/Open Source Software (FLOSS) project has been evaluated in the past through the number of commits made to its configuration management system, number of developers and number of users. Most studies, based on a popular FLOSS repository (SourceForge), have concluded that the vast majority of projects are failures. This study's empirical results confirm and expand conclusions from an earlier and more limited work. Not only do projects from different repositories display different process and product characteristics, but a more general pattern can be observed. Projects may be considered as early inceptors in highly visible repositories, or as established projects within desktop-wide projects, or finally as structured parts of FLOSS distributions. These three possibilities are formalized into a framework of transitions between repositories. The framework developed here provides a wider context in which results from FLOSS repository mining can be more effectively presented. Researchers can draw different conclusions based on the overall characteristics studied about an Open Source software project's potential for success, depending on the repository that they mine. These results also provide guidance to OSS developers when choosing where to host their project and how to distribute it to maximize its evolutionary success.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An Online Monitoring Approach for Web Service Requirements Abstract-Web service technology aims to enable the interoperation of heterogeneous systems and the reuse of distributed functions in an unprecedented scale and has achieved significant success. There are still, however, challenges to realize its full potential. One of these challenges is to ensure the behavior of Web services consistent with their requirements. Monitoring events that are relevant to Web service requirements is, thus, an important technique. This paper introduces an online monitoring approach for Web service requirements. It includes a pattern-based specification of service constraints that correspond to service requirements, and a monitoring model that covers five kinds of system events relevant to client request, service response, application, resource, and management, and a monitoring framework in which different probes and agents collect events and data that are sensitive to requirements. The framework analyzes the collected information against the prespecified constraints, so as to evaluate the behavior and use of Web services. The prototype implementation and experiments with a case study shows that our approach is effective and flexible, and the monitoring cost is affordable. W EB services are Internet-based software applications published using standard interface description languages and universally available via XML-based communication protocols. While this technology is widely expected to enable the interoperation of heterogeneous systems and the reuse of distributed functions, the industry uptake of this technology has been slow [1] . Some research has revealed that the lack of quality assurance and guarantee, thus, leading to deviation of service behavior from requirements, is one of the important factors [2] .. Software requirements are about the expected behavior of the target software in some environment. For serviceoriented software, the behavior of service is not only determined by the program itself, but also affected by many other factors, e.g., hardware, network, and even the client requests. For example, limited hardware resources or low network bandwidth tend to cause a long response time, too many client requests may bring about low performance, and malicious requests may even lead to the denial of service, or wrong result. All these unexpected results are deviations of Web service behavior from the requirements.. Thus, to be able to monitor (i.e., extract and analyze) certain runtime information about these behavior-related factors is an important issue in ensuring the behavior of Web services consistent with requirements.. Many traditional technologies, such as fault tolerance, transaction, and security assurance, are approaches to ensure the behavior of software systems consistent with requirements. In recent years, some similar but more general approaches have been proposed, e.g., autonomous computing [3] , adaptive software [4] , and self-managed system [5] . Monitoring is the first step to all these approaches, where the further actions such as decision making and system adaptation are all based on the information extracted through monitoring. This paper has introduced an online monitoring approach for Web service requirements. It is capable of extracting relevant information about a Web service and its operating environment, and providing support for checking the service behavior against its requirements (in the form of constraints). The proposed approach begins with the pattern-based constraint specification in a new language, WSCDL. The language supports the specification of value and event constraints. The proposed monitoring model covers a wide range of system events and states, and systematically identifies the locations for deploying monitors. The monitoring framework is able to use the formally specified constraints to generate automatically monitoring code. A process guide for how to use the proposed framework is also introduced. Based on a prototype implementation of the framework and a case study, we have conducted a series of experiments to assess effectiveness and impact of our approach. The experiment results have shown that our approach is effective and flexible, and the efficiency and monitoring cost are affordable.. Monitoring systems are usually static. That means, once a monitoring system is deployed, it will not change, and has the same life cycle as the target system. In the proposed framework, the deployed probe can be managed (add, remove, update) online, with the help of online evolution capabilities.. As mentioned in Section 1, monitoring is only the first step for quality assurance. Our ongoing work includes: 1) enhancing the analysis ability of the framework; 2) mining request patterns from the execution of Web service, new constraints from administration operations; and 3) extending our work in online updating [32] to include other online quality assurance capabilities such as online tuning and online migration. To ascertain that the behavior of Web services meets their requirements, one may make assumptions about their operating environment and test them before it is deployed, so as to rectify defects of the system. But it is well known that testing can only tell us there was some defect, never reach a conclusion of "no defect." Furthermore, many environmental factors cannot be emulated realistically during test. Online monitoring and consequent remedial actions to deal with system behavior deviation from requirements should thus be considered as the last barrier of defense against software defects. Considering the distributed nature of client programs and unreliable nature of networks, the importance of monitoring is more pronounced for Web services, compared with traditional software systems.. Let us consider an online Auction Service, which provides a virtual place for selling and bidding items. It has two kinds of clients: the sellers and the bidders. The sellers can register with the service (by sending message "opRegisterRequest"), login to the system (by sending message "opLoginRequest"), and publish the information about the item on sale (by sending message "opPublishRequest"). The bidder can register, login to the system (by sending the same aforementioned messages), bid for an article (by sending message "opBidRequest"), or retract a bid (by sending message "opRetractRequest"). Fig. 1 shows part of the WSDL description of the Auction Service, listing only the description about operation "opBid," which involves messages "opBidRequest" and "opBidResponse." The complete description of the auction system is available at www.sei.pku.edu.cn/~wqx/mass/ auction-example.. WSDL specifies only the operation signature of a service (abstract definition) and location information (concrete part). The research community has widely recognized the need for richer service description to provide more information about a service, e.g., its interaction protocols [13] . In the Auction Service, besides the service signature, there are some important constraints that request messages must follow: Constraint 1. The parameter "password" of "opRegisterRequest" is an instance of type "String," and consists only of letters "a-z" and numbers "0-9," and the length is not longer than 16 and not shorter than 8.. Constraint 2. All parameters that are instances of the type "Price" must be greater than 0 and are instances of the type "float" with two fraction digits.. Constraint 3. The interval between two "opBidRequest" messages from the same bidder should be longer than one second.. Constraint 4. For each bid session, all the "opBidRequest" messages can only be sent if the "opLoginRequest" message has been sent.. These constraints reflect some of the original requirements for the Auction Service example.. To monitor the Auction Service, we should formally describe these constraints first. Then, the monitoring mechanism can check the extracted system behavior information against them automatically. We will introduce our approach to formal constraint specification in the next section.. A constraint may be concerned with system behavior involving different points of interest in the service. For the Auction Service, the aforementioned four constraints are quite different: Constraint 1 is about one message parameter which has type "String"; Constraint 2 is about all message parameters of a given type; Constraint 3 is about the time interval between request messages; Constraint 4 concerns the order of different request messages. Of course, they are only part of all potential constraints. How can we identify the monitoring points systematically? We need a model that identifies and classifies the factors for monitoring, just like that quality models were widely used to help us to analyze the software quality [14] . Such a monitoring model is the focus of Section 4.. Monitoring mechanisms rely heavily on the implementation of the target system. Even for one specific target software and for some specific constraint, multiple monitoring mechanisms may be available. For example, to verify the value "password" in constraint 1, we can insert verifying code directly into the implementation procedure of Web services. We can also use existing mechanisms of the system software (Operation system or Middleware). With Apache Axis, for example, the Handler that processes SOAP messages can be used to obtain the value of "password." (Handlers are also used to support security, transaction, etc., in Web services.) The extracted information can be checked against the constraint inline, or sent to one central analyzer for analysis. Section 5 deals with these issues. Software monitoring has been explored for a long time in various fields, such as performance evaluation and enhancement, fault tolerance, and autonomic computing [6] , [9] , [30] , [31] . Based on these efforts, Delgado et al. [7] proposed a taxonomy and catalog for runtime monitoring based on a comprehensive survey of the area, which analyzed and summarized research concerned with monitoring of traditional software. The specific characteristics of Web service that differs a lot from traditional software (such as dynamism and loose coupling) have posed additional issues for investigation when implementing Web service monitoring. While some work [36] focuses on monitoring of nonfunctional requirements, our work together with [8] , [10] , [11] , [12] , [13] , and [35] focuses on monitoring of functional requirements. As followed, our work is compared with these works in three aspects.

O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B B O O O O O O O B O O O O O O O O O O B O O O O B O O O O O O O B O O O O B O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O B O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O


Parallelisation of genetic algorithms for the 2-page crossing number problem Genetic algorithms have been applied to solve the 2-page crossing number problem successfully, but since they work with one global population, the search time and space are limited. Parallelisation provides an attractive prospect to improve the efficiency and solution quality of genetic algorithms. This paper investigates the complexity of parallel genetic algorithms (PGAs) based on two evaluation measures: Computation-time to Communication-time and Population-size to Chromosomesize. Moreover, the paper unifies the framework of PGA models with the function P GA(subpopulation size, cluster size, migration period, topology), and explores the performance of PGAs for the 2-page crossing number problem. The simplest graph drawing method is that of putting the vertices of a graph on a line and drawing the edges as half-circles. Such drawings are called book drawings, and they correspond to the linear VLSI design. Edge crossing minimisation is the main goal in the linear VLSI design, since smaller number of crossings means cheaper design. The minimal number of edge crossings over all book drawings of a graph is called the book crossing number (29) .. In the 2-page book drawing one places the vertices of a graph G along a line called spine and every edge is completely drawn in one of two pages. The smallest number of crossings over all 2-page drawings of G is called the 2page crossing number of G, denoted by ν 2 (G). Figure 1 shows sample 2-page drawings of K 6 , for which ν 2 (G) = 3, with five ( Fig. 1 (a) ) and three crossings ( Fig. 1 (b) ). Equivalently, the vertices can be put on a circle and the edges can be drawn as straight lines coloured by two colours. The 2-page crossing number is the same as the minimal number of crossings of edges with the same colour. The problem is NP-hard (26) . In contrast, the book thickness problem is to minimise the number of pages used so that edges embedded on the same page do not intersect. Genetic algorithms (GAs) have proved to be good global optimizers for a broad range of optimisation problems, and they have been used successfully for drawing graphs (3; 14; 22; 28) . Moreover, Kapoor et al. presented a genetic algorithm for the book thickness problem in (25) .. Cimikowski (9) tested eight different heuristic algorithms where the order of vertices was determined by finding a Hamiltonian cycle. Hence, contrary to our genetic algorithm, in his tests the order of vertices was always fixed. Recently, Winterbach (33) proposed heuristics for the 2-page crossing numbers and applied them to estimating the plane crossing number of some small complete multipartite graphs. A tabu algorithm was used to produce a good arrangement of vertices, and then either GreedySide algorithm or the neural network of Cimikowski and Shope (10) was used to find a good distribution of edges. Our genetic algorithm finds a good vertex order and edge distribution directly, and we got the same results for the complete multipartite graphs tested by Winterbach, except for a single graph (22) .. Parallelisation can significantly improve the performance of genetic algorithms (1; 2; 15; 16; 17) , and parallel genetic algorithms are easy to implement (5) . One of the most popular tools for parallel computing is Message Passing Interface (MPI) (34) , which provides flexible Cartesian virtual topologies of processors.. In this paper we analyse the complexity of three basic models of parallel genetic algorithms: master-slave, fine-grained, and coarse-grained models. We also examine the effect of different topologies and parameters on the performance of PGAs by testing them on the benchmark graph library (Rome graphs (35) ) and on a special kind of graphs (Xtrees), when solving the 2-page crossing problem.. 2 Sequential genetic algorithm solving the 2-page crossing problem. We have described sequential genetic algorithms (SGA) for the 2-page crossing problem in (22) . The first population of solutions is generated randomly. Fitness, as a measure of quality of solutions, is used to select the better solutions from the current population. The selected solutions undergo the operations of crossover and mutation in order to create a population of new solutions (the offspring population). The process is repeated until the termination criteria given by the user are met.. One of the most important issues is to abstract the characteristics of the problem to make the problem well-fitted for the genetic approach. According to the 2-page crossing number problem, we have defined the four most important aspects of genetic algorithms as described below.

O O O O O O B I I O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O


The Adequacy of Languages for Representing Interaction Mechanisms This paper presents criteria for the adequacy of languages to represent interaction mechanisms. It then uses these criteria to analyse the adequacy of UML. We focus on the interaction mechanisms provided by Web Services technology and by CORBA for request/response, callback, polling and (multicast) message passing.We argue that the criteria for adequacy of a design language are that the language should: (i) be expressive enough to represent the mechanisms; (ii) be easy to use when expressing them; (iii) be platform independent in the sense that it does not force implementation decisions for a mechanism; and (iv) behave corresponding to the mechanisms that it represents. We show that these criteria follow logically from the use of a design language in the design process.For UML we evaluate the first three criteria in a qualitative manner. To evaluate the fourth criteria, we present Coloured Petri Nets that capture the behaviour of both the mechanisms precisely and the UML constructs that represent them. Subsequently, we check the correspondence of their behaviour. Middlewares are defined to make the lives of developers easier, by providing re-usable implementations of advanced interaction mechanisms. Examples of such mechanisms are: remote procedure calls, transactions, publish/subscribe mechanisms, negotiations and long-running business-to-business interactions. Similarly, design languages could make the lives of designers easier, by providing re-usable design concepts that represent these advanced interaction mechanisms. Such design concepts help to: -simplify designs, by providing a single concept, or a small collection of concepts to represent an advanced interaction mechanism;. -transform designs to implementations, by providing abstract (platform independent) concepts of which the transformation to (various) middlewares is clear; and -analyse the correctness of the design in early stages of the design process.. To be able to analyse the correctness of a design the design concepts must properly reflect the relevant properties of the represented interaction mechanisms. In contrast, if concepts do not reflect the properties of their middleware counterparts faithfully, this may lead to wrong conclusions during analysis. For example, the concept of reliable message passing does not behave like message passing middleware, in which message passing is typically unreliable. This may cause designers to conclude wrongly that a message will always be received by the receiving side and therewith that the implemented business transaction will be successfully completed. This leads to problems if the concept is used in designs in which reliability is an issue (such as the design of a banking system).. The first goal of this paper is to present and motivate the criteria for adequacy of advanced interaction design concepts (and design concepts in general) and to show how these criteria can be checked. The second goal of the paper is to evaluate the adequacy of UML for representing advanced interaction mechanisms, based on the criteria.. To motivate the criteria for adequacy of design concepts, we analyse the role that design concepts play in the design process.. To analyse the adequacy of UML for representing advanced interaction mechanisms, we analyse interaction mechanisms implemented in existing middleware and the concepts that UML 2.0 [19, 20] provides to represent interaction mechanisms. To capture their properties precisely, we define both the interaction mechanism and the concepts using Coloured Petri Nets. Subsequently, we evaluate the adequacy of the design concepts from UML 2.0 for representing the analysed interaction mechanisms, based on the criteria that are the result of the first step.. We focus on the mechanisms that CORBA [22] and Web Services [29] provide for request/response, callback, polling and (multicast) message passing.. This paper is further structured as follows. Section 2 explains the basic properties of a design process and motivates the criteria that design concepts in the design process should meet. Sections 3 and 4 analyse interaction mechanisms in Web Services and CORBA respectively, and model them in terms of Coloured Petri Nets. Section 5 analyses the UML 2.0 model elements to represent interactions and models them in Coloured Petri Nets. Section 6 evaluates if and how the interaction mechanisms from sections 3 and 4 can be represented using UML 2.0. Section 7 presents related work and section 8 discusses our conclusions and future work. In this paper we provided criteria for adequate interaction design concepts. We also provided an in depth analysis of the interaction mechanisms implemented by Web Services and CORBA and the UML 2.0 model elements for representing these interaction mechanisms.. Based on the criteria and the analysis of UML and the interaction mechanisms, we evaluated the adequacy of UML for representing these mechanisms. From this evaluation, we conclude that UML is unfaithful with respect to the representation of communication failure and unsuitable for representing polling and message passing mechanisms. This means that UML is not adequate for the design of reliable systems and for systems that use interactions other than synchronous request/response, callback and one-to-one message passing.. In the context of the project in which this work is embedded, we aim to define concepts that are adequate for representing advanced interaction mechanisms (such as one-to-many message passing, transactions and negotiation) and reliable systems. In this paper we focused on design at the lowest level of abstraction before choosing a particular middleware platform (Web Services or CORBA). In future work we will also consider higher levels of abstraction and middleware platforms and develop concepts for those levels. Also, we will develop concepts to represent other aspects of interaction mechanisms, such as: threading mechanisms and creation and destruction of bindings between communicating parties (e.g. event channel subscriptions). There is a long history of research towards concepts to represent interactions at various stages in a design process. Interaction concepts have been studied in the area of reference models (such as [14, 15] ), design languages (such as [6, 8, 10, 13, 19, 20, 25, 27] ) and architectural description languages (such as [1, 17] ). Our work aims to contribute to these areas by evaluating and improving the concepts that are defined in these areas.. More recently, interaction patterns are being studied [3, 11, 26] . Interaction patterns represent frequently occurring compositions of interactions, mainly in the context of business interaction. The project in which the work presented in this paper is embedded, aims to contribute to this area by defining the implementation relation between these business interactions and interactions that can support them in existing middleware.. Platform specific languages (typically UML profiles) exist to graphically represent Web Services and CORBA interactions. The CORBA profile for UML [23] is an example of such a language. Our work complements this work, because we do not aim to define a graphical notation to represent interactions, but to define concepts that define the behaviour of those interactions precisely. Languages that represent the behaviour of Web Services or CORBA based applications include [4, 9] . However, this work focuses on the behaviour in which interactions are used, while we focus on the behaviour of the interactions themselves.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Linear programming based algorithms for preemptive and non-preemptive RCPSP Abstract: In this paper, the RCPSP (Resource Constrained Project Scheduling Problem) is solved using a linear programming model. Each activity may or may not be preemptive. Each variable is associated to a subset of independent activities (antichains). The properties of the model are first investigated. In particular, conditions are given that allow a solution of the linear program to be a feasible schedule. From these properties, an algorithm based on neighbourhood search is derived. One neighbour solution is obtained through one Simplex pivoting, if this pivoting preserves feasibility. Methods to get out of local minima are provided. The solving methods are tested on the PSPLIB instances in a preemptive setting and prove efficient. They are used when preemption is forbidden with less success, and this difference is discussed. We consider in this paper the Resource Constrained Project Scheduling Problem (RCPSP). A project is composed of activities and subject to precedence and resource constraints, and the objective is minimizing the makespan. A precedence constraint consists in completing some activity before another one can start. The resource constraints specify that each activity requires constant amounts of renewable resources while being processed, these resources having limited capacities.. The approach developed in this paper involves variables associated to subsets of activities that can be simultaneously processed during the schedule. A time-indexed linear formulation of the non-preemptive version of the RCPSP involving these feasible subsets (here called valid antichains) has been introduced by Mingozzi et al. [7] : precedence and non-preemption constraints can be simply formulated in spite of a very large number of variables (depending on the length of the discretized time interval [0; T ], where T is an upper bound of the makespan, and on the number of valid antichains of the project). In such approaches, the resource constraints are taken into account in the computation of these antichains, which allows to eliminate these constraints from the linear formulation.. By evading the non-preemption constraints and partially the precedence constraints, Mingozzi et al. also formulate an LP-relaxation of the RCPSP, which was slightly modified by Brucker and Knust [3] and then Baptiste and Demassey [1] to give the best known lower bounds of the problem. They both implement a destructive approach on T and use constraint propagation techniques cooperating with the resolution of this linear relaxation through column generation.. Concerning the preemptive version of the RCPSP (an activity can be interrupted and later resumed without penalty), it is assumed in papers that look 2 The valid antichain based model This paper presents a characterization of the solution set for the preemptive and non-preemptive RCPSP, based on a linear programming model. An exploration algorithm of this solution space has been developed, in order to visit as many local minima as possible. The experimental results are very good in the preemptive case. The gap is much larger for non-preemptive activities. This is due to the difficulty for a given improving column to pass the feasibility test. Other combinatorial problems may be formulated as linear programs with additional conditions on the column generation. Our method should behave well for these, provided the feasibility test is not too demanding, as in the preemptive case.

B I O O O O O O O O O O O O O B I I B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O


Evolving Multimodal Networks for Multitask Games Abstract-Intelligent opponent behavior makes video games interesting to human players. Evolutionary computation can discover such behavior, however, it is challenging to evolve behavior that consists of multiple separate tasks. This paper evaluates three ways of meeting this challenge via neuroevolution: 1) multinetwork learns separate controllers for each task, which are then combined manually; 2) multitask evolves separate output units for each task, but shares information within the network's hidden layer; and 3) mode mutation evolves new output modes, and includes a way to arbitrate between them. Whereas the first two methods require that the task division be known, mode mutation does not. Results in Front/Back Ramming and Predator/Prey games show that each of these methods has different strengths. Multinetwork is good in both domains, taking advantage of the clear division between tasks. Multitask performs well in Front/Back Ramming, in which the relative difficulty of the tasks is even, but poorly in Predator/Prey, in which it is lopsided. Interestingly, mode mutation adapts to this asymmetry and performs well in Predator/Prey. This result demonstrates how a human-specified task division is not always the best. Altogether the results suggest how human knowledge and learning can be combined most effectively to evolve multimodal behavior. The results for Front/Back Ramming and Predator/Prey are presented in this section in terms of the metrics described above. Front/Back Ramming is described first because its results are more straightforward, followed by Predator/Prey, for which the results were more surprising. Domains involving multiple tasks are common in both video game research and robotics research. Therefore, various approaches have been implemented to deal with such domains. This related work is divided into approaches that involve separately evolved controllers, single controllers with modular architectures, and controllers that achieve functional modularity via recurrency.

O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Edge-to-Edge Measurement-based Distributed Network Monitoring Continuous monitoring of a network domain poses several challenges. First, routers of a network domain need to be polled periodically to collect statistics about delay, loss, and bandwidth. Second, this huge amount of data has to be mined to obtain useful monitoring information. This increases the overhead for high speed core routers, and restricts the monitoring process from scaling to a large number of flows. To achieve scalability, polling and measurements that involve core routers should be avoided. We design and evaluate a distributed monitoring scheme that uses only edge-to-edge measurements, and scales well to large network domains. In our scheme, all edge routers form an overlay network with their neighboring edge routers. The network is probed intelligently from nodes in the overlay to detect congestion in both directions of a link. The proposed scheme requires significantly fewer number of probes than existing monitoring schemes. Through analytic study and a series of experiments, we show that the proposed scheme can effectively identify the congested links. The congested links are used to capture the misbehaving flows that are violating their service level agreements, or attacking the domain by injecting excessive traffic. Continuous monitoring of a network domain is necessary to ensure proper operation of the network by detecting possible service violations and attacks. Attackers can impersonate a legitimate customer by spoofing flow identities. Network filtering [15] at routers can detect such spoofing if the attacker and the impersonated customer are in different domains. Otherwise, the attacks remain undetected. The quality of service (QoS) enabled networks face QoS attacks. In this setting, the attacker is a regular user of the network trying to get more resources (a better service class) than what it has signed (paid) for. A QoS network provides different classes of service for different cost, which can entice attackers to steal bandwidth and other network resources. Such attacks involve injecting traffic into the network with the intent to steal bandwidth or to cause QoS degradation, by causing other customers' flows to experience longer delays, higher loss rates, and lower throughput. Taken to an extreme, this may result in a denial of service (DoS) attack.. A large variety of network monitoring tools can be found in [17] . Many tools use SNMP [9] , RMON [28], or NetFlow [11] , which are built-in functionality for most routers. Using these mechanisms, a centralized or decentralized model can be built to monitor a network. The centralized approach to monitor network latency, jitter, loss, throughput, or other QoS parameters suffers from scalability. One way to achieve scalability is to use a hierarchical architecture [2, 27] . Subramanyan et al [27] design a SNMP-based distributed network monitoring system that organizes monitoring agents and managers in a hierarchical fashion. Both centralized or decentralized models obtain monitoring data by polling each router of a network domain, which limits the ability of a system to scale for large number of flows. The alternative way of polling is to use an event reporting mechanism that sends useful information typically in a summarized format only when the status of a monitored element changes. A more flexible way of network monitoring is by using mobile agents [20] or programmable architecture [3] . However, periodic polling or deploying agents in high speed core routers put non-trivial overhead on them. We propose a very low overhead monitoring scheme that does not involve core routers for any kind of measurements. Our assumption is that if a network domain is properly provisioned and no user is misbehaving, the flows traversing through the domain should not experience a high delay or a high loss. An excessive traffic due to attacks changes the internal characteristics of a network domain. This change of internal characteristics is a key point to monitor a network domain.. Edge-to-edge monitoring scheme is studied in [16] , which devises a network monitoring mechanism to detect attacks on QoS domains using network tomography [6, 14] . This monitoring mechanism measures the service level agreement (SLA) parameters, and compares these measurements with the values negotiated between a service provider and a user. To infer SLA parameters, the tomography-based scheme constructs a tree from the network topology, and probes the leaves from the root. Probing from the root to all leaves can not infer SLA parameters in both directions of a link. This can be achieved with much higher overhead. Our goal in this work is to devise a low overhead monitoring scheme that can detect attacks in both directions of all links in a network domain.. The proposed monitoring scheme has two phases. In the first phase, we continuously measure edge-toedge link delays to observe any unusual delay pattern. All ingress routers (entry points) sample the incoming traffic to probe delay of the paths followed by a user packet. This measures the delay experienced by a user inside the domain. If the delay is higher than a pre-defined threshold (SLA value), the edge routers conduct intelligent probing for loss measurements. For this probing, an overlay network is formed using all edge routers on top of the physical network. The probing does not calculate loss ratio for each individual link, instead, the congested links with high losses are identified using edge-to-edge loss measurements. We provide two methods to identify congested links: simple method and advanced method. In the simple method, all edge routers probe their neighbors in clockwise and counter-clockwise direction. This method requires only O(n) probing, where n is the number of edge routers. Through extensive analysis, both analytical and experimental, we show that the simple method is very powerful to identify the congested links to a close approximation. We provide an advanced method that searches the topology tree intelligently for probes that can be used to refine the solution of simple method, if necessary. When the network is less than 20% congested the advanced method requires O(n) probes. If the congestion is high, it requires more probes, however, does not exceed O(n). In the second phase of our monitoring process, we use the congested links as a basis to identify edge routers through which traffic are entering into and exiting from the domain. From exiting edge routers, we identify the flows that are violating any SLA agreement. If the SLA is violated for delay and loss, the network is probed to detect whether any user is stealing bandwidth. The service violations can indicate a possible attack on the same domain, or on a downstream domain. In case of a DoS attack, numerous flows from different sources are destined to a victim. These flows aggregate on their way as they get closer to the victim. Monitoring an upstream network domain can detect these high bandwidth aggregates that could result in DoS attacks on downstream domains [16, 21] . To control the attacks, filters are activated at edge routers through which flows are entering into a network domain.. Using simulation, we conduct a series of experiments to evaluate the proposed monitoring scheme. We conclude that the distributed monitoring scheme shows a promise for an efficient and scalable monitoring of a domain. This scheme can detect service violations, bandwidth theft attacks, and tell when many flows are aggregating towards a downstream domain for a possible DoS attack. The scheme requires low monitoring overhead, and detects service violations in both directions of any link in a network domain.. The rest of the paper is organized as follows: The related work is discussed in Section 2. Measuring all necessary network parameters for monitoring purposes is presented in Section 3. This section discusses our proposed monitoring scheme, and analyzes its strength and limitations. Section 4 explains how to use the monitoring scheme to detect service violations and DoS attacks. Experimental results and discussions are provided in Section 5. We conclude the paper in Section 6. We have proposed a distributed network monitoring scheme to keep a domain safe from service violations and bandwidth theft attacks. The network is monitored continuously for unusual high delay pattern. When a delay is high, each edge router probes its neighbors in the overlay tree created on top of the physical network. In our monitoring scheme, we do not measure actual loss of all internal links, instead, we identify all congested links with high losses with edge-to-edge measurements. We provide a simple solution to detect congested links. This method requires fewer probes and identifies congested links when the network is sparsely congested. In other cases, the simple method provides a close approximation of congested links instead of the exact number of congested links. To refine the solution of simple method, an advanced method is proposed. Our solution outperforms the existing monitoring mechanism because they require fewer probes to detect attacks in both directions of a link. The number of probes required for the proposed way of monitoring is significantly low (probe traffic is 0.002% of link capacity for an OC3 links). The simulation results indicate that the proposed scheme detects service violations, bandwidth theft attacks, and DoS attacks caused by flow aggregation towards a victim network domain.. [24] G. Sager. Security fun with OCxmon and cflowd. Intenet2 working group meeting, Nov. 98.. [ Figure 19 : Intersection of probe paths P and P . If R 2 is an edge router R 2 → R 3 and R 2 → R 4 do not exist.. Proof: Lemma 1. Let the core router c is connected to only two other routers R 1 and R 2 . No probe path can be constructed that either includes the link R 1 → C and does not include C → R 2 or vice versa; or includes R 2 → C and does not include C → R 1 or vice versa. The traffic that passes through the link R 1 → C also passes through C → R 2 . The traffic that passes through the link R 2 → C also passes through C → R 1 . Therefore, for the purpose of probing, a logically equivalent overlay network can be constructed by removing C and replacing the links R 1 → C and C → R 2 with an equivalent single link R 1 → R 2 . R 2 → C and C → R 1 can be replaced by R 2 → R 1 . We say that the link R 1 → R 2 is congested if and only if at least one of the links R 1 → C and C → R 2 is congested, i.e. the bandwidth of R 1 → R 2 is the minimum of the bandwidths of R 1 → C and C → R 2 . Similarly, the bandwidth of R 2 → R 1 is the minimum of the bandwidths of R 2 → C and C → R 1 . 2. Proof: Lemma 4. Let the link R 1 → R 2 ( Figure 19 ) appears in path P in the first round of probing and path P in the second round of probing. If R 2 is a core router, it is connected to at least two other routers, say R 3 and R 4 (Lemma 2). P passes through the link R 2 → R 4 and P passes through the link R 2 → R 3 . Since the tree does not have any cycle, P and P never meet again. If R 2 is an edge router, both P and P terminates at R 2 . Therefore, P and P can not have any common link in their paths after node R 2 . Similarly, it can be shown that P and P can not have common links before they meet at node R 1 . That is |P ∩ P | ≤ 1. 2 One common way of monitoring is to log packets at various points throughout the network and then extract information to discover the path of any packet [24] . This scheme is useful to trace an attack long after the attack has been accomplished. The effectiveness of logging is limited by the huge storage requirements especially for high speed networks. Stone [26] suggested to create a virtual overlay network connecting all edge routers of a provider to reroute interesting flows through tunnels to central tracking routers. After examination, suspicious packets are dropped. This approach also requires a great amount of logging capacity.. Many proposals for network monitoring [5, 12] give designs to manage the network and ensure that the system is operating within desirable parameters. In efficient reactive monitoring [12] , the authors discuss ways to monitor communication overhead in IP networks. Their main idea is to combine global polling with local event driven reporting to monitor a network. Breitbart et al [5] identify effective techniques to monitor bandwidth and latency in IP networks. The authors present probing-based techniques where path latencies are measured by transmitting probes from a single point of control. They describe algorithms for computing an optimal set of probes to measure latency of paths in a network, whereas we focus on measuring parameters using distributed agents.. In [10] , a histogram-based aggregation algorithm is used to detect SLA violations. The algorithm measures network characteristics on a hop-by-hop basis, uses them to compute end-to-end measurements, and validates end-to-end SLA requirements. In large networks, efficient collection of management data is a challenging task. While exhaustive data collection yields a complete picture, there is an added overhead. The authors propose an aggregation and refinement based monitoring approach. The approach assumes that the routes used by SLA flows are known, citing VPN and MPLS [8] provisioning. Though routes are known for double-ended SLAs that specify both ingress and egress points in the network, they are unknown in cases where the scope of the service is not limited to a fixed egress point. Like RON [4] , we check violations using average values in a recent time frame. This reduces constraints on the network setup, and the need for knowledge of the set of flows traversing each router. Duffield et al [13] use packet "stripes" (back-to-back probe packets) to infer link loss by computing the correlation of packet loss within a stripe at the destinations. This work is an extension of loss inference for multicast traffic, described in [1, 7] . The stripe-based probing mechanism is adopted to monitor loss characteristics inside a domain without relying on the core routers [16] . To infer loss, a series of probe packets, called a stripe, are sent from one edge router to two other edge routers with no delay between the transmissions of successive (usually three) packets. For example, in a two-leaf binary tree spanned by nodes 0, k, R 1 , R 2 , stripes are sent from the root 0 to the leaves to estimate the characteristics of one link, say k → R 1 (Figure 1 ). The first two packets of a 3-packet stripe are sent to R 2 and the last one to R 1 . If a packet reaches any receiver, we can infer that the packet must have reached the branch point k. If R 2 gets both packets of a stripe, it is likely that R 1 will receive the last packet of the stripe. The transmission probability A k for node k can be calculated knowing how many packets are sent from the root and how many of them received by both receiver. Using A k and number of packets reach to R 1 and R 2 , we can calculate the successful transmission probability of the link k → R 1 . Similarly, a complementary stripe is sent to estimate the characteristics of link k → R 2 . By combining estimates of stripes down each such tree, the characteristics of the common path from 0 → k is estimated. This inference technique extends to general trees by sending probes from root to each ordered pair of leaves [13] . Ji and Elwalid [18] show that measurement-based monitoring using tree is scalable when the probe packets reach the edge routers with high probability. If the internal loss is very high, the solution does not scale for a large network.. The unicast probing scheme is extended in [16] for routers with active queue management. This scheme is used to monitor loss inside a QoS network domain. The stripe-based monitoring scheme requires less overhead than core-based monitoring. In this paper, we propose a scalable scheme that requires much less probes than stripe-based scheme. Kim et al [19] collect statistical data from every single router for each service class and then analyze the data to compute edge-to-edge QoS of aggregate IP flows. This approach has very high overhead, and not suitable for real time monitoring.

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Efficient interactive configuration of unbounded modular systems Interactive configuration guides a user searching through a large combinatorial space of solutions to a system of constraints. We investigate a class of very expressive underlying constraint satisfaction problems: modular recursive constraint systems of unbounded size. A precomputation step is used to obtain a configuration algorithm for such systems that supports the user efficiently with bounded response time. This precomputation step determines all solutions for each module, which are computed and stored in compact data structures such as Binary Decision Diagrams (BDDs), in order to eliminate run-time search. The precomputation step also detects ill-behaved module collections that have no finite solutions. The runtime interaction algorithm scales well as its response time only depends on the amount of the information passed locally between the modules, and not on the size of the entire configured structure. Our algorithm was implemented and tested on an industrial example, and gives good response times. We believe this is the first known sound and complete algorithm for solving unbounded interactive configuration problems, with bounded response time per interaction.

O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O


Computing

in

Coq

with

Infinite

Algebraic

Data

Structures

Abstract.

Computational

content

encoded

into

constructive

type

theory

proofs

can

be

used

to

make

computing

experiments

over

concrete

data

structures.

In

this

paper,

we

explore

this

possibility

when

working

in

Coq

with

chain

complexes

of

infinite

type

(that

is

to

say,

generated

by

infinite

sets)

as

a

part

of

the

formalization

of

a

hierarchy

of

homological

algebra

structures.

One

main

feature

of

constructive

type

theory,

through

the

well-known

CurryHoward

isomorphism,

is

the

equivalence

between

proving

and

programming.

This

is

clearly

one

of

the

advantages

of

Coq

with

respect

to

other

proof

assistants,

like

Isabelle/HOL

This

characteristic

is

the

base

of

reflective

tactics,

pioneered

by

S.

Boutin

and

successfully

used,

for

instance,

in

Computing

can

play

another

role

when

formalizing

a

proof.

It

can

be

useful,

for

example,

to

check

some

conjecture

over

concrete

cases.

When

dealing

with

standard

data

structures

(as

lists,

trees,

and

the

like),

these

experiments

can

be

done

in

a

parallel

line

by

programming

the

tests

in

Java,

C,

or

any

other

programming

language.

If

infinite

data

structures

occur,

programming

them

is

a

more

delicate

task,

and

it

can

be

rewarding

to

keep

a

tighter

link

among

programs

and

specifications..

Infinite

data

structures,

presented

as

coinductive

objets

as

streams,

have

been

dealt

with

in

the

theorem

proving

literature

(see

for

instance).

In

this

work,

we

undertake

another

via

to

manage

the

infinity,

working

with

algebraic

structures

of

infinite

type

(that

is

to

say,

generated

by

infinite

sets)

We

report

in

this

paper

on

an

experiment

of

this

nature,

in

the

area

of

homological

algebra.

It

is

well-known

that

homological

information

is

not

computable

over

general

(infinite

type)

chain

complexes

(see

For

instance,

if

(C,

d)

is

an

acyclic

chain

complex,

and

x

C

n

is

a

cycle

(this

means

d

n

(x)

then

there

exists

z

C

such

that

d

(z)

x

(that

is,

x

is

a

boundary).

But

if

C

is

a

free

module

of

infinite

type,

and

no

other

information

is

available,

there

is

no

general

algorithm

computing

a

pre-image

z

of

x..

Sergeraert's

effective

homology

is

a

theory

allowing

solving

large

classes

of

problems

of

this

sort,

even

in

the

infinite

dimensional

case.

This

paper

continues

our

previous

work

in

translating

Sergeraert's

ideas

to

theorem

provers

with

the

aim

of

formalizing

this

part

of

algorithmic

mathematics

and,

more

importantly,

of

applying

formal

methods

to

the

study

of

the

Kenzo

system

(a

Common

Lisp

program

developed

by

Sergeraert

to

implement

effective

homology

algorithms).

The

first

important

milestone

in

this

area

was

the

mechanized

proof

in

the

Isabelle/HOL

proof

assistant

of

the

Basic

Perturbation

Lemma

(BPL),

published

in

This

formal

proof

was

carried

out

in

the

Higher

Order

Logic

(HOL)

built

on

top

of

Isabelle,

and

therefore

extracting

programs

from

it

was

not

a

simple

task.

The

findings

on

this

topic

were

reported

in

A

different

approach

is

being

carried

out

by

T.

Coquand

and

A.

Spiwack

who

are

using

Coq

to

model

a

part

of

Category

Theory,

and

then

trying

to

obtain

a

BPL

proof

in

this

larger

context..

The

data

structures

of

effective

homology

are

organized

in

two

layers

(as

algebraically

modeled

in

the

first

layer

is

composed

of

algebraic

data

structures

(chain

complexes,

simplicial

sets,

and

the

second

one

of

standard

data

structures

(lists,

trees,

which

are

representing

elements

of

data

from

the

first

layer.

Infinite

type

data

structures

appear

only

in

the

first

layer.

Computing

in

this

first

layer

can

be

done

in

an

abstract

way,

and

it

is

equivalent

in

Coq

to

proving

theorems.

For

example,

a

theorem

stating

"the

direct

sum

of

two

chain

complexes

is

a

chain

complex"

contains

an

algorithm

constructing

the

mentioned

direct

sum.

Coq

can

deal

with

this

structure,

no

matter

whether

it

is

of

finite

or

infinite

type.

But

actual

computations

really

take

place

within

algebraic

structures

of

the

first

layer.

To

compute

with

Coq

in

this

sense

has

no

advantage

of

being

any

more

direct.

It

is

needed

to

construct

concrete

instances

of

chain

complexes

and

other

possibly

infinite

algebraic

data

structures.

Then

we

must

build

concrete

elements

(second

layer)

of

these

particular

structures,

and

finally

put

to

work

the

algorithms

abstractly

described

in

the

first

layer..

In

this

paper

we

discuss

this

procedure

in

a

case

related

to

the

effective

homology

of

the

cone

of

a

chain

complex

morphism.

This

formalization

was

part

of

the

implementation

in

Coq

of

the

algorithm

computing

the

effective

homology

of

a

bicomplex

(see

Now,

we

use

the

computing

capabilities

of

Coq

to

explore

whether

some

concrete

cones

are

acyclic

or

not,

as

a

previous

step

to

proving

a

general

property..

The

paper

is

organized

as

follows.

Section

contains

some

preliminaries

on

algebraic

structures,

both

in

Mathematics

and

in

Coq.

Section

describes

the

formalization

in

Coq

of

the

algorithm

computing

the

effective

homology

of

a

cone,

in

a

way

that

slightly

generalizes

our

previous

work

in

Concrete

Coq

instances

of

chain

complexes

of

infinite

type

are

introduced

in

Section

Then

explicit

calculations

with

elements

are

presented

in

Section

using

Coq

as

a

computing

tool

to

check

some

conjectures.

The

paper

ends

with

conclusions,

future

work,

and

the

bibliography.

The

Coq

source

files

are

available

at

documentos/CCIADS.zip.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O