Time Representations and Mathematical Models for Process Scheduling Problems During the last 15 years, many mathematical models have been developed in order to solve process operation scheduling problems, using discrete or continuous time representations. In this paper, we present a unified representation and modeling approach for process scheduling problems. Four different time representations are presented, compared, and applied to single-stage and multi-stage batch scheduling problems, as well as crude-oil operations scheduling problems. We introduce three solution methods that can be used to achieve global optimality or obtain near-optimal solutions depending on the stop criterion used. Computational results show that the Multi-Operation Sequencing time representation is superior to the others as it allows efficient symmetry-breaking and requires fewer priority-slots, thus leading to smaller model sizes. Rigorous optimization of real-world problems are often based on advanced programming tools such as mixed-integer linear programming (MILP) or constraint programming (CP, see Rossi et al. (2006) ). These tools rely on a mathematical or symbolic representation of the problem which is applied by an end-user. In some cases, the relationship between the problem description and its mathematical model is not clear. Therefore an intermediate step is included in the optimization approach (see Figure 1 ). In this step, the representation used is detailed and approximations are made. For instance, in the context of scheduling problems, using a discrete-time formulation is in general a constraining approximation of the actual problem, and thus, it may lead to a suboptimal solution as discussed in Floudas and Lin (2004) .. Additionally, it is important to note that several mathematical models may be used to obtain the global optimal solution of the problem, which is the best possible solution according to a given optimization criterion. For example, many continuoustime representations rely on a specific parameter representing the number of time points (Kondili et al., 1993) , time intervals (Lee et al., 1996) , or event points (Ierapetritou and Floudas, 1998) used. Therefore, the scheduling problem is represented by an infinite set of mathematical models, one for each possible value of this parameter (all positive integers). The global optimal schedule is the best solution among the optimal solution of all these models. In general, it is not possible to know a priori the parameter value that will lead to the global optimal solution, although it is sometimes possible to derive upper and lower bounds for it. The common trade-off is that global optimality may be guaranteed with a large value of this parameter which often results in prohibitive solution times.. Many different time representations have been introduced to solve scheduling problems (for review see Floudas and Lin (2004) ). Experience has shown that, depending on the characteristics of the problem, some time representations are better suited than others. In this paper, we focus on scheduling problems which rely on: a) a set of possible operations, or actions, that can be performed once, several times, or not at all; b) scheduling decisions that involve both selecting, parametrizing and sequencing the operations that should be executed; c) scheduling constraints such as release dates, due dates, bounds on processing times, non-overlapping constraint, cardinality constraints, and precedence constraints; d) additional side constraints that are used to model more complex features such as limited inventory management or process constraints.. It should be noted that, for example, the selection of operations may correspond to the selection of equipment or discrete resources for tasks in a state-task-network (Kondili et al., 1993) or in a resource-task-network (Pantelides, 1994) . In general, operations are defined by fully disaggregating all possible discrete selections of actions in the scheduling system. In contrast, parameterization of operations corresponds to continuous decisions such as batch sizes, transfer volumes, or process operating conditions.. The main objective of this work is to develop a unified modeling approach for scheduling problems in order to facilitate the evaluation of several time representations, both in terms of computational time and solution quality. First, a simple scheduling problem is introduced as an example. Next, we study four different types of time representations, which have been used in the literature and clarify the relationships between them. Then, basic MILP models for pure scheduling constraints are presented for each of these time representation. Using concepts from graph theory (cliques and bicliques), we show how these models can be generally strengthened based on the structure of the scheduling problem. Two solution methods are then developed to solve these mathematical formulations. Finally, three types of problems are presented and solved using the different approaches in order to show the effectiveness of the strengthened formulations and to provide elements of comparison between the different time representations. In this paper, we presented four different time representations that in different forms have been previously introduced to solve chemical scheduling problems. Using the common concept of priority-slot, it is shown that it is possible to derive relationship results between these time representations. Additionally, generic scheduling constraints are developed as well as specific solution algorithms for each model. We apply these models to batch scheduling of single and multi-stage plants, and to crude-oil operations scheduling.. Intuitively, the representation that requires the least priority-slots is the most computationally effective. In practice, the MOS time representation proves to be superior to the other time representations on the three types of problem tested due to the smaller number of priority-slots and symmetry-breaking constraints used. The discrete-time MOS-FST time representation behaves well on problems with constant processing times while the MOS-SST time representation performs better with variable processing times. Although it requires a larger number of priority-slots, the SOS time representation is comparable to the MOS-SST time representation for the crude-oil operations scheduling problems. In particular, the SOS model is efficiently solved using DFA-based symmetry-breaking constraints (see Mouret et al., 2009) .. A natural extension of this work is to adapt the time representations and corresponding mathematical models to more complex problems that require precise tracking of operation start and end events. Besides, instead of directly solving the models using an MILP solver, multi-stage algorithms can be developed in order to decompose the different decision levels, such as selection and sequencing of operations.

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Advanced hands-on training for distributed and outsourced software engineering Today's software projects are often distributed across multiple locations. This distribution poses new challenges produced by the cooperation across different countries, times zones, and cultures. Software engineering courses have to prepare students accordingly. This paper reports an experience on teaching a distributed software engineering course. In this course, students develop software in collaboration with five universities located in Italy, Hungary, Russia, Switzerland, and Ukraine. The projects allow students to face the difficulties of developing software in a globalized context, and provide a practical experience on distributed software engineering. We describe the major obstacles on organizing such a course, and we suggest best practices to achieve successful outcome. Developing software engineering projects is difficult. Distributing the team across different countries, continents, time zones and cultures does not help to solve the problem; it makes it harder. Due to the restriction in communication, software requirement specifications, interface specifications, and project management become a key to develop successful distributed projects. Failures reported in outsourced and distributed projects happened not due to lack of technical expertise, but due to lack of proper management. One of the roles of software engineering courses is to prepare students to face the new challenges of distributed software engineering.. Most of today's software engineering courses include a student project as key component. The project provides a practical experience for the students. Distributed software engineering courses should also include a project, in this case a distributed project. To provide a real experience the teams in the distributed projects should be located in different countries with different cultures, native languages and if possible, different time zones. How can one organize these projects?. Some of the difficulties of organizing these projects, besides the time schedules of the courses, which can differ from one country to another, is the coupling between the teams: the success of the project does not only depend on the success of a local team but also the success of the team located in another country. This coupling does not exist in typical software engineering projects where a project is developed by one team. Another problem is the integration of the projects in a final system: even if the local teams perform a great job, the project might still fail. This paper describes a practical experience teaching distributed software engineering. This experience recreates the atmosphere of an international project, and faces some of the severe obstacles that may come up in a real project. Some aspects of the course have been described in previous publications [7] [5]; the novel elements in this paper are: (1) a description of the major obstacles to organizing distributed projects, (2) an assessment of the results, and (3) best practices to achieve successful results. The paper is organized as follows: in Section 2 we describe our software engineering course. Section 3 presents the assessment. The paper concludes with lessons learnt. One of the lessons learnt is how to organize distributed projects in an academic environment. If it is the first time the course includes a distributed project, it is strongly recommended that the teaching stuff at the different universities meet to discuss the organization of the course. Important items to discuss are the project topic, the programming language to use, communication tools, set up for svn or cvs repositories, and student assignments and deadlines. We find useful to provide a scope document and a basic project implementation. The scope document should describe the overall project structure, and the interaction between the groups.. Organizers in each university should ensure that students commit to the project. In many universities students can leave the course at any time; this is strongly undesirable for a distributed project. We have addressed this problem working with small groups of volunteer students. In some universities, students are selected. If students leave the course during the project, we reorganize the groups.. To implement the project, we recommend that a common programming language is chosen. This language can be also used to define a common interface specification between the subsystems. If the students are not familiar with the programming language to use, it is useful to organize an introductory training before the project starts.. Communication in distributed project is difficult. However, it is not enough only to describe the problem of communication in a lecture. Before starting the projects, we run several communication exercises, and we discuss the results in an exercise session. For example one good game can be found in student books for the official Microsoft MSF course. Furthermore, we require students write a communication plan . T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 Keeping the students motivated and excited about the project is important for all software engineering projects. Before starting each phase (1-requirements, 2-interface specification, and 3-implementation), students filled in a questionnaire about how motivated they were. They were asked to give a number from 1 to 10 where 1 is no motivation and 10 very motivated. The questionnaire was filled in by 100% of the groups (the questionnaire was part of the assignments, so all groups reported their motivation). In average, the motivation was very high (8. 2) at the beginning of phase 1. Before phase 2 and phase 3, motivation was still very high: 8.1 and 8.0 respectively. We think challenging projects help to enhance the motivation of students.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


E-learning multimedia applications: Towards an engineering of content creation Abstract: In the same manner that e-learning applications are becoming increasingly important at the university, there are still some critical questions that should be solved with the objective of making use of the potential offered by current Web Technologies. The creation of contents that are able of capturing the attention of interest of the students and their disposal in an appropriate way constitute the main purpose of this work. The teaching content engineering expounded shows the different stages that should form part of the process. A development team, composed of different professional profiles, will work together with the lecturers of the subject to which the contents are been created, i.e. multimedia videos and interactive applications. This process should be developed according to a methodology that assure the use of appropriate resources, all that tasks -suitable of being-should be modularized and factorized. This paper presents the acquired experience in the development and use of multimedia contents for e-learning applications, created for some of the subjects of the degree in computer science engineering. The deliveries of these contents make use of Internet and video streaming techniques. The result of the work shows the students satisfaction, including their comments. During recent years University Education has experienced an important progress with respect to the resources used for lecturing. Techniques have changed from traditional blackboard methods to the use of projector transparencies (slices) and, later on, the use of video projector and PowerPoint to present the content of the lessons. Nowadays, the use of these technologies has become more common. As far as content availability for the student is concerned, it has passed from traditional notes taken in classrooms, books and photocopies to digital format of books or notes taken in classrooms, which are available in the subject's web page. In fact, the universities use to place courses on the web focusing on information delivery rather than learning. The proposal of European Higher Education Area provides the framework to take an important step forward in techniques used in lectures and in the methodology used to create them. For this reason, new methodological approaches are introduced in order to value an active learning [1] as opposed to a traditional passive learning. To value the effort that students should make in order to assimilate knowledge and not only consider the number of hours of attendance to classes. The use of e-learning techniques makes this easier; its wide range of application allows increasing education quality and delivery of information.. E-learning multimedia applications: Towards an engineering of content creation 117 allows the quality of education and its availability to increase [3] . The creation of multimedia contents for e-learning could be developed together with other European Universities, and its use could be shared within the same knowledge areas. This allows a new common line of work to be opened. It will be possible to promote European cooperation to guarantee the quality of higher education using comparable methodologies and criteria. New information technologies and e-learning will be indispensable tools in lecturing in the near future. The increasing bandwidth available for Internet connection and multimedia capacities found in current computers allow them to be used as a completely valid way for teaching. However, the lack of multimedia content production is a very important cause that limits an e-learning widespread. The creation of large-scale teaching content for lecturing needs a production methodology to assure the optimization of resources and, therefore, a reduction in costs. This work intends to contribute with the methodological lines applied in the production of multimedia teaching content for lecturing. New teaching methodologies require new teaching aims; the role of lecturers in the new educational process should be restated in the whole educational process. Lecturers should reduce the time dedicated to teaching lessons using traditional blackboard methods. This activity could be mainly, replaced by e-learning techniques. Lecturers could dedicate time to activities that offer a better quality in teaching, transmitting motivation about the subject being studied and directing the students in their studies [10] . The future of teaching in the European society should see an improvement in the quality of education, its availability, and a lower cost for students as well as for educative organizations. The obtaining of an optimum workflow [5] and the use of resources by the different production tasks is the main purpose of the proposed methodology. For that, we define different production phases.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Absolute and specific measures of research group excellence Abstract A desirable goal of scientific management is to introduce, if it exists, a simple and reliable way to measure the scientific excellence of publicly-funded research institutions and universities to serve as a basis for their ranking and financing. While citation-based indicators and metrics are easily accessible, they are far from being universally accepted as way to automate or inform evaluation processes or to replace evaluations based on peer review. Here we consider absolute measurements of research excellence at an amalgamated, institutional level and specific measures of research excellence as performance per head. Using biology research institutions in the UK as a test case, we examine the correlations between peer-review-based and citation-based measures of research excellence on these two scales. We find that citation-based indicators are very highly correlated with peerevaluated measures of group strength but are poorly correlated with group quality. Thus, and almost paradoxically, our analysis indicates that citation counts could possibly form a basis for deciding on how to fund research institutions but they should not be used as a basis for ranking them in terms of quality. The problem of measuring the scientific excellence of research institutes and universities is one of current and continued importance [1] . It is important to be able to detect and support the most promising research groups and to have sound and robust bases to wisely plan and invest for the future. Metrics and indicators of scientific activity may be based on the volume of researchers, the number of published papers, the amount of citations to these publications, the number of PhD students and on finances generated by application of scientific ideas, amongst other factors.. It is often argued that automated scientometric or bibliometric indicators may be an inadequate substitute or proxy for peer-review based evaluations of the merit of research institutes [2] . There is a lot of arguments against the use of indicators in isolation, suggesting they reflect only one aspect of the research and should only be used as an adjunct to peer review (for example, see [3, 1] ). But in spite of the problems with indicators and metrics, reasonable and cheap approaches to the evaluation of scientific productivity and quality are desirable for practical purposes, not least because institutional peer review is an expensive and timeconsuming exercise [1] .. Here we investigate "absolute" and "specific" measurements of scientific excellence on the scale of university research groups. E.g., the absolute citation count for a department is the total number of citations, irrespective of how many researchers that department contains. The corresponding specific citation count is then the average number of citations per head (see, for example, [4, 5] ). In physics parlance these correspond respectively to the notions of "extensive" and "intensive" quantities and are usually represented mathematically by upper-case and lower-case symbols. That convention is borrowed here for absolute and specific measurements. In this paper we compare these specific and absolute notions of quality and strength with indicators of citation impact, to determine if the latter may be used as a reliable proxy for the former. We consider the overall strength of a research collective (a university department, research centre, group or some variant, component or amalgam of these) as measured through a peer-review evaluation scheme to be an absolute quantity and denote it by S. A peer-review measure of the quality 1 s of a group of researchers is then the strength per head. Similarly, the total citation impact I is considered here as the absolute impact of research group while its average value i (calculated per head) corresponds to specific impact. Thus, "absolute" refers to total institutional measurements while "specific" means average properties per individual. We then define strength as "volume of quality" and absolute impact as "volume of average impact".. In this paper, to make a quantitative comparison we use data from the British system of evaluation of research and education (Research Assessment Exercise) and from Evidence, a company within Thomson Reuters, one of the world's leading providers of scientific information. Using biology research groups and departments in the UK as a test case, we show that the citation-based measure i is not a good proxy for the peer-review measure s, in that these two specific measures are rather poorly correlated. However, when scaled up to the actual size of the department N , the absolute citation impact I = iN is very strongly correlated with the overall strength S = sN as measured by peer review. (Here and below N means the number of researchers in group.) This means that citations, if used in an informed manner, could possibly be used as a proxy for departmental or group strength particularly for large groups, but should not be as an estimate of research quality. When applied to large research groups or departments in particular, this may offer an alternative or at least complement to peer review.. 1 Background: The research assessment exercise and the normalised citation impact. For the last quarter century, the Research Assessment Exercise (RAE) provided the premier yardstick for the measurement of scientific performance of research institutions in United Kingdom. This introduced an explicit and formalized process to assess the quality of research [8] . Six evaluation exercise have occurred since the first one in 1986 and evaluation frameworks of this type now constitute the single most important event in the UK research calender each five or so years. For the purpose of the 2008 exercise, each academic discipline was assigned to one of 67 units of assessment (UOA). In order to receive quality related funding any Higher Education Institution (HEI) could make submissions to RAE in any UOA. Using published criteria, RAE experts assess these submissions and generate graded profile for each of them. These profiles quantify the proportion of a department's or research centre's work which falls into each of five quality bands as follows [8]:. -4*: Quality that is world-leading in terms of originality, significance and rigour; -3*: Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence; -2*: Quality that is recognised internationally in terms of originality, significance and rigour; -1*: Quality that is recognised nationally in terms of originality, significance and rigour; -Unclassified -Quality that falls below the standard of nationally recognised work or work which does not meet the published definition of research for the purposes of the assessment.. An example of a quality profile is given in Table 1 . Besides obvious benefits in terms of prestige, marketing and publicity potential, the research quality profiles determined how much funding each university department receives in the years following the exercise. Funding is determined by a formula combining the weighted quality scores of individual research group or centres. The formula is subject to regional and temporal variation, but the one introduced by the Higher Education Funding Council for England (HEFCE) immediately subsequent to RAE 2008 was [10]. Here pn * represents the percentage of a team's research which was rated n * and s may be considered as a single specific measure of a university's overall research quality in a particular discipline. The total amount of quality related funding distributed by HEFCE to a given university post RAE is proportional to the overall strength of the submission. where N is the number of researchers submitted from a given university. Thus, strength S is an absolute measure of performance. Notwithstanding various managerial tactics and manoeuvres, the RAE, and similar exercises in other countries, is considered to be reasonably reliable since it is based on peer-review evaluation. Despite its many drawbacks and limitations, peer review is considered by the academic community at large to be the most reliable and trustworthy scheme to evaluate the worth of curiosity-driven research, in particular. On the other hand, peer-review-based evaluation is expensive and time-consuming process. In addition, the very act of measuring the scientific system distorts the very process it purports to measure. This is Goodhart's Law, a type of socioeconomic counterpart of the Heisenberg uncertainty principle in physics. Therefore, it is reasonable to propose some alternatives to the RAE. Evaluations based entirely upon citation counts are some of many possible candidate schemes (see, for example, [11, 12] ). Some authors advocate substitution of the RAE by citation counting due to claimed good correlation between the resultant rankings obtained [11, 13] . But there are also authors (e.g., [14] ) who point to different weaknesses of citation analysis such as self-citing, bias, technical errors connected with citation data retrieval or different typical rates of citations for various disciplines. Although methods have been, and are being, developed to try to deal with these phenomena, no universally satisfactory automated system has yet emerged.. So, the question arises as to how good citation-based indicators can be as proxies for peer review. Since the last UK exercise generated quality measurements in far more detail than previous RAEs, and since subsequent evaluation frameworks in the UK will amalgamate some of the UOA's used in 2008 into fewer, larger subject groupings, RAE 2008 provides a window of great detail in which answers to this question may be sought. Having precise numbers assigned to each institution for the 67 disciplines it is possible to perform accurate comparisons between RAE .0 ‡ An overall quality profile is constructed by summing the three separate weighted components ("Output", "Environment" and "Esteem") using a special cumulative rounding methodology which avoids unfair consequences that simple rounding can produce [9] .. peer-review evaluation and citation scores. Thus we arrive at the main questions we wish to address herein: how do RAE 2008 scores correlate with citation rates?. As already noticed before, to get the citation-based measure, we use the data provided by Evidence. This company offers a service analysing research performance tailored to individual client requirements [15] . We consider the so-called normalised citation impact (NCI) i used by Evidence as a coefficient of departmental performance in a given discipline.. Evidence calculate the NCI using data from Thomson Reuters databases [3, 16] . Similarly to Relative Citation Rate (RCR) (i.e., [17] ), NCI is a relative measure as it is calculated by comparing to a mean or expected citation rate. On the other hand, this is a specific measure of citation impact because it is averaged to be a measure of impact for research group. It has long been known that citation rates are different for different disciplines. The main advantage of Evidence calculations is non-trivial normalisation of citation counts between different academic disciplines. To achieve this, the total citation count for each paper can be normalised to an average citations per paper for the year of publication and either the field or journal in which the paper was published (the so-called "rebasing" of the citation count). The normalised value is known as the NCI [16] . Only the four papers per individual which were submitted to RAE 2008 were taken into account by Evidence in order to calculate the average NCI for research groups [16] . The RAE does not emplot a comparable normalisation process and this is a serious and acknowledged weakness of that peer evaluation exercise [18] .. Thus, the NCI may be considered as a specific measure of the research impact of a department in a given field and we denote it by i. The corresponding absolute measure of impact (the total impact of the department or group) is denoted by I. The relationship between the two is. In the following, we compare the specific indicators of quality and impact s and i as measures of the average strength and impact of the group or department per individual contained within it. We also compare the counterpart absolute measurements S and I as measures of the overall strength and total impact of the group as a whole. We use the biology research sector in the UK as a test case. We show that while the specific measures s and i are only weakly correlated, the absolute ones S and I are strongly aligned. The analysis thus suggests that citations form a poor basis for measuring institutional research quality but could form a good basis for measuring strength. Since quality related funding as defined by HEFCE is proportional to departmental strength rather than quality, the citation-based measures may offer a reasonable and cost effective way to decide on funding while ameliorating the negative effects of Goodhart's law. We have to note that although RAE assessments as well as the Evidence results are used for practical purposes, they are still subject to debate within the scientometric community. This is an additional reason to compare them. A goal of scientometrics is to develop a method to provide a reliable measurement of scientific excellence using minimal efforts. However, any attempts to replace current peer-evaluation based systems have to be done carefully and robustly. Quality related funding, such as that administered by HEFCE post RAE in the UK is proportional to absolute measures of research strength since financial support for research obviously needs to be weighted according to group size. Therefore, a citation based approach, at least for large groups, may be introduced as less intrusive and more cost effective alternative to national peer review. This would also have the advantage of ameliorating the distortions to the research system introduced on a national basis every five years or so through Goodhart's phenomenon, although one might expect a citation-based system to introduce distortions of its own.. However, we have also seen that the citation metrics used here are not well correlated with peer-review measures of group research excellence. Since the latter are used to rank institutions, it is clear that citation counts are not a good basis on which to make such comparisons.. Thus we arrive at the almost paradoxical conclusion that the citation-based metric (NCI) may be used as an excellent proxy for peer-reviewed measurements of institutional scientific strength but it is only a poor proxy for measures of quality. Since quality related funding is strength based, this may offer a much cheaper alternative to the system currently in use in the UK and some other countries.. The analysis presented here is based on biology research groups in the UK. Further analyses are underway for other disciplines to determine the broader suit-ability of scientometric measures of specific and absolute research-group properties. The nuances connected with peculiarities of groups of different sizes should be also studied. Since in the UK system, a large group with, say 40 members, provides 40 × 4 = 160 submissions, which may be sufficiently large for statistical fluctuations to be ironed out. But the RAE evaluation of a small group, say with 5 staff members, is based on the statistics of only approximately 5 × 4 = 20 submissions. This is far more susceptible to inaccuracies and statistical noise. So, any automated evaluation of large groups is much more robust than for small groupseven if one is only interested in measuring strength rather than quality. A possible way to balance this, if introducing citation counts, would be to keep peer review for small and medium sized groups or to require a greater number of outputs per person for such groups to improve statistics. Investigations into such schemes are underway. In any case, it is clear from the current test case that any attempt to automatically evaluate research quality should very carefully take account of group or departmental sizes [6] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Feature Selection for Gene Expression Using Model-Based Entropy Gene expression data usually contain a large number of genes but a small number of samples. Feature selection for gene expression data aims at finding a set of genes that best discriminate biological samples of different types. Using machine learning techniques, traditional gene selection based on empirical mutual information suffers the data sparseness issue due to the small number of samples. To overcome the sparseness issue, we propose a model-based approach to estimate the entropy of class variables on the model, instead of on the data themselves. Here, we use multivariate normal distributions to fit the data, because multivariate normal distributions have maximum entropy among all real-valued distributions with a specified mean and standard deviation and are widely used to approximate various distributions. Given that the data follow a multivariate normal distribution, since the conditional distribution of class variables given the selected features is a normal distribution, its entropy can be computed with the log-determinant of its covariance matrix. Because of the large number of genes, the computation of all possible log-determinants is not efficient. We propose several algorithms to largely reduce the computational cost. The experiments on seven gene data sets and the comparison with other five approaches show the accuracy of the multivariate Gaussian generative model for feature selection, and the efficiency of our algorithms. In this paper, we suggest multivariate Gaussian generative models for feature (gene) selection because multivariate normal (Gaussian) distributions are a maximum-entropy probability distribution. Using the model-based entropy estimation, we avoid the data sparseness problem that commonly happens in the empirical information gain approach.. Using the properties of multivariate normal distributions, we derive the feature selection methods based on the D-optimality criterion and its approximation, the A-optimality criterion.. To efficiently select genes from gene expression data, where the numbers of features are large and the numbers of samples are relatively small, we propose several simple algorithms (a few lines of code). Among them, Algorithm 4 and Algorithm 7 are most suitable for gene expression data. The time complexity of the proposed algorithms is linear to the product of the number of genes and the number of samples for each iteration of selection.. The experiments on seven gene data sets and the comparison with other five approaches show the accuracy and efficiency of our approach. His research has been focused on probabilistic modeling, Bayesian inference, Gaussian processes, general machine learning algorithms, and their applications to information retrieval, text mining, recommender systems, clinical data analysis, intrusion detection, and visual recognition. He is the author of more than 40 research papers in international conferences and journals and has served as a program committee member or reviewer for conferences like ICML, NIPS, SIGIR, and ECML.. Tao Li received the PhD degree in computer science from the Department of Computer Science, University of Rochester, in 2004. He is currently an associate professor in the School of Computer Science, Florida International University, Miami. His research interests are data mining, machine learning, information retrieval, and bioinformatics.. Yihong Gong received the BS, MS, and PhD degrees in electronic engineering from the University of Tokyo in 1987, 1989, and 1992, respectively. He then joined the Nanyang Technological University of Singapore, where he worked as an assistant professor in the School of Electrical and Electronic Engineering for four years. From 1996 to 1998, he worked at the Robotics Institute, Carnegie Mellon University, as a project scientist. He was a principal investigator for both the Informedia Digital Video Library Project and the Experience-on-Demand Project funded in multimillion dollars by US NSF, DARPA, NASA, and other government agencies. In 1999, he joined NEC Laboratories America and has been leading the Multimedia Processing Group since then. In 2006, he became the site manager to lead the Cupertino branch of the laboratories. His research interests include multimedia content analysis and machine learning applications. The major research achievements from his group include news video summarization, sports highlight detection, data clustering, and Smart-Catch video surveillance that led to a successful spin-off.. . For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib. Generally two types of feature selection methods have been studied in the literature: filter methods [17] and wrapper methods [16] . Filter-type methods are essentially data preprocessing or data filtering methods. Features are selected based on the intrinsic characteristics that determine their relevance or discriminative powers with regard to the target classes. In wrapper-type methods, feature selection is "wrapped" around a learning method: the usefulness of a feature is directly judged by the estimated accuracy of the learning method. Wrapper methods typically require extensive computation to search for the best features. As pointed out in [32] , the essential differences between the two methods are 1. that a wrapper method makes use of the algorithm that will be used to build the final classifier while a filter method does not, and 2. that a wrapper method uses cross validation to compare the performance of the final classifier and searches for an optimal subset while a filter method uses simple statistics computed from the empirical distribution to select an attribute subset. Wrapper methods could perform better but would require much more computational costs than filter methods. Most gene selection schemes are based on binary discrimination using rank-based filter methods [8] such as t-statistics and information gain [31] . One critical issue in these rank-based methods is data sparseness, and most of the rank-based methods do not take redundancy into consideration. In order to remove the redundancy among features, a Min-Redundancy and Max-Relevance (mRMR) framework is proposed in [25] . Yu and Liu [36] also explored the relationship between feature relevance and redundancy and proposed a method that can effectively remove redundant genes. In addition, ReliefF, a widely used feature subset selection method, has also been applied to gene selection [22] . In the paper [19] , a hierarchical Bayesian model is used to enforce the sparsity in the features. However, the sparsity is indirectly controlled by a parameter, not specified by the given number of features to select.. In Section 5, we will describe several gene selection methods used in our experimental comparisons in detail. In this paper, we propose a model-based approach to estimate the information gain, instead of on the data itself. This would overcome the limitations of data sparseness. In addition, the model parameters can be explicitly and efficiently estimated via a few matrix operations. We conduct three sets of experiments using seven data sets as described in Section 6.1. In the first set of experiments, we compare the classification accuracy of data with gene selection and without gene selection using Support Vector Machine (SVM) classifiers implemented in the LIBSVM package [4] . The second set of experiments provides a comprehensive study on the performance of different gene selection methods under different conditions. In the third set of experiments, we discuss the number of selected genes.

B I O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Multiproperty-Preserving Domain Extension Using Polynomial-Based Modes of Operation Abstract-In this paper, we propose a new double-piped mode of operation for multiproperty-preserving domain extension of message authentication codes (MACs), pseudorandom functions (PRFs), and pseudorandom oracles (PROs). Our mode of operation performs twice as fast as the original double-piped mode of operation of Lucks while providing comparable security. Our construction, which uses a class of polynomial-based compression functions proposed by Stam, makes a single call to a 3n-bit to n-bit primitive f 1 at each iteration and uses a finalization function f 2 at the last iteration, producing an n-bit hash function 2 3 ) query complexity as long as 1 and 2 are public random functions. To our knowledge, our result constitutes the first time (2 ) unforgeability that has been achieved using only an unforgeable primitive of -bit output length. (Yasuda showed unforgeability of (2 5 6 ) for Lucks' construction assuming an unforgeable primitive, but the analysis is suboptimal, as noticed by us and others; in this paper, we also show how Yasuda's bound can be improved to (2 ).) In related work, we strengthen Stam's collision resistance analysis of polynomial-based compression functions (showing unforgeability of the primitive suffices) and discuss how to implement our mode by replacing 1 with a 2 -bit key blockcipher in Davies-Meyer mode or by replacing 1 with the cascade of two 2 -bit to -bit compression functions.Index Terms-Adaptive preimage resistance, double-piped mode of operation, hash function, indifferentiability, message authentication code (MAC). T HE Merkle-Damgård transform has been the most popular method to build a cryptographic hash function from a fixed-size compression function. A major advantage of this construction is that it preserves collision resistance with an appropriate padding algorithm, allowing one to focus on the construction of secure compression functions. However, Joux [17] showed that if computing collisions becomes somehow feasible for the underlying compression function, then the hash function may fail worse than expected: for a hash function based on a compression function of -bit output, one can find a -multicollision only with complexity, which is much smaller than required for an ideal random function. This observation led to several generic attacks such as long-message second preimage attacks [19] and herding attacks [18] . Lucks observed that these weaknesses can be mitigated by increasing the size of the internal state and claimed that the internal state size should be seen as a security parameter of its own right [24] . Since a secure compression function of a larger output size might be harder to construct than the hash function itself, Lucks proposed to use a "narrow" compression function in a double-piped mode. In a subsequent paper [35] , Yasuda rigorously analyzed the security of the double-piped mode of operation as a multiproperty-preserving domain extension. Specifically, he showed that Lucks' double-piped mode of operation preserves unforgeability up to query complexity, and indistinguishability and indifferentiability both up to query complexity. Moreover, it was later noticed by several researchers that Yasuda's unforgeability bound could be increased to with a slightly modified proof (see Section VII). As such, Lucks' construction turned out to provide nearly optimal security. However, the fact that Lucks' compression function uses two applications of a (fairly strong) primitive remains a drawback. Stam [32] , [33] recently proposed a class of wide-pipe compression functions making a single call to an equivalent primitive (we call these polynomial-based compression functions). In this paper, we analyze the security properties of double-piped modes using Stam's polynomial-based compression functions, focusing on MAC-preservation, PRFpreservation, and PRO-preservation. Except for PRO-preservation (where we only achieve security), our bounds are comparable to those found by Yasuda for Lucks' original construction (and even better for unforgeability, given the sub-optimality of Yasuda's bound in that case, though the "corrected" unforgeability bound exceeds ours by a factor of ) even though our construction has twice the rate. 1 Besides performance, a second concern that arises for Lucks' double-pipe construction is the rather strong primitive it assumes: a -bit to -bit function (note that careful consider-ation is typically already given for the construction of -bit to -bit compression functions from smaller or more available primitives). Here, we also tackle this problem and show that our double-piped polynomial-based mode can be implemented with a blockcipher of -bit key in Davies-Meyer mode, in either the ideal-cipher model or the weaker "unpredictable cipher" model (see Section V) without significant loss of security. We also prove MAC-preservation and PRF-preservation for a compression function obtained by replacing the -bit to -bit primitive with the cascade of two -bit to -bit primitives. This latter result potentially opens the door to implementing the compression function with two calls to an -bit key blockcipher in Davies-Meyer mode (which would be the first time, to our knowledge, that a -bit to -bit compression function using two calls to an -bit key blockcipher is proved secure nearly up to the birthday bound). There currently seems to be no particular obstacle to achieving the latter result, except for the tediousness of the foreseeable analysis. All the compression functions discussed in this paper, including the cascaded and blockcipher variants, were proposed by Stam [32] , [33] . In [33] , Stam proves the collision resistance of polynomial-based compression functions of degrees 2 and 3 in the random function model, and also proves the collision security of the quadratic blockcipher mode in the ideal cipher model. Here, our contribution is that we weaken the model by showing that collision resistance is already assured when and are unforgeable/unpredictable rather than random. (It is this weakening of the model that allows us to prove MACpreservation results for the resulting hash functions.) Regarding the quadratic cascade compression function, Stam proves collision resistance for a special class of nonadaptive adversaries assuming random primitives. Our analysis supports fully adaptive adversaries and once again weakens the model to unforgeable primitives.. Lucks [25] recently proposed a double-pipe hash function iterating a -bit to -bit compression function which, like the quadratic blockcipher-based mode, uses a single call to a blockcipher of -bit key. However, by contrast to the quadratic blockcipher compression function, Lucks' compression function is neither collision resistant nor preimage resistant. As a consequence, collision and preimage security can only be proved in the iteration (higher security notions like indifferentiability are unaddressed). On the other hand, for Lucks gives a better explicit collision security bound than we do for the quadratic blockcipher compression function: versus queries, respectively. As the first to formally study multiproperty-preserving domain extension, we refer to [2] and [3] . In particular, Bellare and Ristenpart [2] study domain extension for MACs, PRFs, and PROs in the dedicated key setting, similar to our analysis. Since then, various constructions for multiproperty-preserving domain extension have been proposed and analyzed [8] , [16] . However, most constructions are using single-piped modes of operation that provide security only up to the birthday bound (particularly in terms of MAC security). This paper can be seen as an extension of Yasuda's work [35] since our main achievement is to double the rate of that construction while maintaining comparable MAC-preservation and PRF-preservation properties. However, from a technical standpoint we owe most to Dodis and Steinberger [10] , whose "multicollision-to-forgery" (MTF) balls-in-bins games are used in nearly all of our analyses (the sole exception being the preimage awareness bound for polynomial-based compression functions). Indeed, the main "message" of this paper may well be the versatility and power of MTF games.. Recently, Dodis and Steinberger [11] have shown how to achieve beyond-birthday domain extension for MACs from noncompressing primitives. Their construction, however, remains mainly of theoretical interest, due to its inefficiency. Fig. 3(a) . In the experiment, an adversary has oracle access to and tries to produce a valid tag for a new message . Here, we call a message "new" if it has not been queried to oracle . The forgery advantage of is defined by. The probability is taken over the random choice of and 's coins (if any). We define as the maximum of over all adversaries making at most queries whose total combined length is at most bits (including the forgery produced by ) and of "running time" at most . The "running time" is defined to be the total running time of the experiment, including the time required to compute the answers to 's queries. We write for if is a family of fixed input length functions, as in this case is automatically determined by .. The WCR of is estimated by the experiment described in Fig. 3(b) . In contrast to the definition of collision resistance (in the dedicated-key setting) where is the provided key , is allowed only oracle access to . Let Then the WCR of , denoted , is defined to be the maximum of over all adversaries making at most queries whose total combined length is at most bits and of running time at most . When is a family of fixed input length functions, we likewise write instead of . Our security proof for unforgeability will follow the approach developed by An and Bellare [1] is the maximum number of queries to a member of required to compute for such that . Indifferentiability and Indistinguishability: In the indifferentiability framework, a distinguisher is given two systems and . Here, is an ideal primitive used as a building block for the construction of . An ideal primitive and a probabilistic Turing machine with oracle access to have the same interfaces as and , respectively. The simulator tries to emulate the ideal primitive so that no distinguisher can tell apart the two systems and with a significant probability, based on their responses to queries that the distinguisher may send. We say that the construction is indifferentiable from if the existence of such a simulator is proved. The indifferentiability guarantees universal composability of : if is indifferentiable from , then can replace in any cryptosystem, and the resulting cryptosystem is at least as secure under the assumption that is ideal as under the assumption that is ideal. For example, if an iterative hash function based on a fixed-size compression function is indifferentiable from a VIL random oracle in the FIL random oracle model, then the iterative hash function can replace the VIL random oracle in any cryptosystem, and the resulting cryptosystem remains secure in the FIL random oracle model if the original scheme was secure in the VIL random oracle model. 3 Now we give a formal definition of indifferentiability in the information-theoretic model. For a more comprehensive introduction of the indifferentiability framework, we refer to [6] and [27] .. Definition 1: A Turing machine with oracle access to an ideal primitive is said to be -indifferentiable from an ideal primitive if there exists a simulator of running time at most with oracle access to such that for any distinguisher making at most queries, it holds that If is a public random function, then is called a -pseudorandom oracle (PRO). If is not allowed to make queries for the underlying primitive, we obtain the definition of indistinguishability.

O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I B I B B I I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O B I I I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A comparison of extrinsic clustering evaluation metrics based on formal constraints Abstract There is a wide set of evaluation metrics available to compare the quality of text clustering algorithms. In this article, we define a few intuitive formal constraints on such metrics which shed light on which aspects of the quality of a clustering are captured by different metric families. These formal constraints are validated in an experiment involving human assessments, and compared with other constraints proposed in the literature. Our analysis of a wide range of metrics shows that only BCubed satisfies all formal constraints. We also extend the analysis to the problem of overlapping clustering, where items can simultaneously belong to more than one cluster. As Bcubed cannot be directly applied to this task, we propose a modified version of Bcubed that avoids the problems found with other metrics.Keywords Clustering Á Evaluation metrics Á Formal constraintsThe clustering task consists of grouping together those objects which are similar while separating those which are not. The difference with classification tasks is that the set of categories (or clusters) is not known a priori.Given a similarity metric between objects, evaluation metrics can be intrinsic, i.e., based on how close elements from one cluster are to each other, and how distant from elements in other clusters. Extrinsic metrics, on the other hand, are based on comparisons between the In this paper, we have analyzed extrinsic clustering evaluation metrics from a formal perspective, proposing a set of constraints that a good evaluation metric should satisfy in a generic clustering problem. Four constraints have been proposed that correspond with basic intuitions about the quality features of a clustering solution, and they have been validated with respect to users' intuitions in a (limited) empirical test. We have also compared our constraints with related work, checking that they cover the basic features proposed in previous related research.. A practical conclusion of our work is that the combination of BCubed precision and recall metrics is the only one that is able to satisfy all constraints (for non-overlapping clustering). We take this result as a recommendation to use BCubed metrics for generic clustering problems. It must be noted, however, that there is a wide range of clustering applications. For certain specific applications, some of the constraints may not apply, and Fig. 23 Counterexample of non-hierarchical overlapping clustering for BCubed and purity-based metrics new constraints may appear, which could make other metrics more suitable in that cases. Some recommendations derived from our study are:. • If the system quality is determined by the most representative cluster for each category, metrics based on matching between clusters and categories can be appropriate (e.g. Purity and Inverse Purity). However, we have to take into account that these metrics do not always detect small improvements in the clustering distribution, and that might have negative implications in the system evaluation/refinement cycles.. • If the system quality is not determined by the most representative cluster for each category, other metric families based on entropy, editing distances, counting pairs, etc. would be more appropriate.. • If the system developer wants to avoid the quadratic effect over cluster sizes (related to our fourth formal constraint), we recommend to avoid using metrics based on counting pairs. Instead of this, the developer may use entropy-based metrics, edit distance metrics or BCubed metrics.. • In addition, if the developer does not want to penalize merging unrelated items in a ''rag bag'' (''other'' or ''miscellaneous'' cluster), then the only recommendable choice is BCubed metrics.. We have also examined the case of overlapping clustering, where an item can belong to more than one category at once. Most evaluation metrics are not prepared to deal with cluster overlaps and its definition must be extended to handle them (the exception being purity and inverse purity) We have then focused on BCubed metrics, proposing an intuitive extension of BCubed precision and recall that handles overlaps, and that behaves as the original BCubed metrics in the absence of overlapping.. As a case study, we have used the testbed from the WePS competitive evaluation task, where purity and inverse purity (combined via Van Rijsbergen's F) were used for the official system scores. A cheating solution, which receives an unreasonably high F score (rank 2 in the testbed), is detected by the extended Bcubed metrics, which relegate the cheating solution to the last position in the ranking. We have seen, however, that BCubed can, in extreme cases, give maximal values to imperfect clustering solutions. This is an evidence that a complete formal study, similar to the one we have performed for the nonoverlapping case, is required.. Three main limitations of our study should be highlighted. The first one is that our formal constraints have been checked against users' intuitions in a limited empirical setting, with just one clustering scenario taken out of a typical ad hoc retrieval test bed, and with a reduced number of users. An extension of our empirical study into different clustering applications should reinforce the validity of our constraints.. The second one is that, beyond formal constraints, there are also other criteria that may apply when selecting a metric. For instance, two important features of any evaluation metric are its ability to scale (v.g. is 0.5 twice as good as 0.25?) and its biases (Strehl 2002 ). While we believe that our constraints help choosing an adequate metric family, more features must be taken into account to select the individual metric that is best suited for a particular application. In particular, it must be noted that hierarchical clustering, which is a wide information access research area, has peculiarities (in particular regarding the cognitive cost of traversing the hierarchical cluster structures) that need a specific treatment from the point of view of evaluation. Our future work includes the extension of our analysis for hierarchical clustering tasks and metrics.. Finally, note that considering the computational properties of evaluation metrics is beyond the scope of this paper, but might become a limitation for practical purposes.. Indeed, the BCubed metric, which is the best according to our methodology, requires an O(n 2 ) computation, which is more costly than computing most other metrics (except those based on counting pairs). While the typical amount of manually annotated material is limited, and therefore computing BCubed is not problematic, this might become an issue with, for instance, automatically generated testbeds. Let us see the results when using BCubed metrics (Table 3) . BCubed Recall behaves similarly to Inverse Purity, ranking the cheat system in first position. BCubed Precision, however, does not behave as Purity. In this case, the cheat system goes down to the end of the ranking. The reason is that BCubed computes the precision of items rather than the precision of clusters. In the cheat system output, all items are duplicated and inserted into a single cluster, increasing the number of clusters. Therefore, the clustering solution provides more information than required, and the overall BCubed precision of the distribution is dramatically reduced (see Sect. 6.2). On the other hand, the BCubed recall slightly decreases (0.99) because the multiplicity of a few items belonging to more than two categories is not covered by the cheat system. 6.5 Is the problem of overlapping clustering solved?. In this article, we have selected BCubed metrics for extending to overlapping clustering tasks because it satisfies all our proposed constraints for the non overlapping problem, and we have obtained a metric that appears to be more robust than purity and inverse purity. Note, however, that we should extend and redefine our set of formal constraints in order to know if we have reached a satisfactory solution to the problem. In fact, our metric has at least one problem: a maximal value of Bcubed does not necessarily imply a perfect clustering distribution. This is a basic constraint that is trivially met by all metrics in the non-overlapping problem, but becomes a challenge when overlaps are allowed. Let us illustrate the problem.. In the case of hierarchical clustering, it is easy to show that if extended Bcubed Precision and Recall are maximal (1), then the distribution is perfect. When Precision and Recall are 1, then if two elements share n clusters they must share n categories. Assuming that the distribution has a hierarchical structure, this is equivalent to saying that two elements share a branch of the hierarchy up to level n. One could build up the tree branching, in each step, according to the lenght of the branches shared by each pair of elements to arrive univocally to the ideal clustering. Therefore, maximal BCubed values imply a perfect distribution. Surprisingly, in the case of non-hierarchical clustering none of the current metrics satisfy this basic restriction. Let us illustrate the problem with the example in Fig. 23 . All clusters are pure (Purity = 1), and for every category there is a cluster that contains all elements belonging to the category (Inverse Purity = 1). In fact, for every category there is a cluster with maximal precision and recall over elements of that category, and therefore the F-measure (see Sect. 4.1) also achieves a maximal value. In addition, BCubed metrics is also maximal, because the three elements appear three times each, and each pair of elements shares three categories and three clusters. And yet the clustering solution is clearly non-optimal.. Purity and Inverse purity fail to detect the errors because they do not consider multiplicity of occurrences in the elements (see Sect. 6). But BCubed metrics also fail in this case, because they only check coherence between pairs of elements, but this can have crossed relations in different clusters in such a way that they satisfy restrictions on paired elements.. Note, however, that generating such a counterexample requires knowing the ideal distribution beforehand, and therefore this problem cannot lead to a cheat system that gets high scores exploiting this weakness of the metrics. In practice, the possibility of having misleading scores from BCubed metrics is negligible.

O O O O B B I O O B I O O O O O O O B I O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Identifying design parameters for fuzzy control of staged ventilation control systems Conventional staged ventilation systems are commonly used in agriculture to maintain interior environments near desired conditions for livestock housing and greenhouses. This paper identifies design parameters for fuzzy-based control of these staged ventilation systems. A simple non-steady state heat balance is used in conjunction with a broiler house simulation model, and coupled with a model for the control system, to simulate control system performance. Difficulties with implementation of conventional staged ventilation control, and the proposed fuzzy inference technique, arise because of the discontinuous nature of these highly non-linear systems. Comparisons between the new fuzzy stage controller and conventional staged control are made. Effects of varying the identified design parameters for the fuzzy stage controller, including different degrees of control precision and energy use, rule base complexity, and the rate of change of house temperature are made. Results indicate that existing staged ventilation control systems which utilize microprocessors could realize significantly enhanced control flexibility by a simple software modification to incorporate the fuzzy staged controller method. Staged ventilation systems are commonly used for environment control in livestock facilities and greenhouses in the USA. These systems utilizes a series of discrete stages of ventilation and heating to add air conditioning capacity as the current inside air temperature deviates from a set point. Such a staged control system is highly non-linear, not readily analyzed with classical control techniques, and yet works quite well in many cases. For smaller facilities with adequate or excessive capacity for ventilation-based energy transfer, and low internal heat and moisture loads, only a few stages are generally necessary. Ventilation stages in these instances provide a balance between energy exchange capacity and the thermal needs of the space's occupants (Zhang and Barber, 1993; Chao and Gates, 1996) . When outside conditions and internal heat loads are such that desired interior conditions cannot be maintained with the equipment at hand, the resultant deviation from set point is considered acceptable in comparison to costs and complexity of a more optimal control technique (Cole, 1980) .. As building sizes have increased, however, there has been a tendency toward a greater number of ventilation stages. This increased stage capacity means that either a larger deviation from set point is accepted for significant periods of operation, or temperature differentials between stages must be reduced. In the latter case, there is a limit to how small the differentials can be made without excessive switching of large ventilation loads. Further, by definition of their operation, such systems require a specified difference between set point temperature and building interior temperature for ventilation to occur; in a sense they behave as 'discrete proportional controllers'. Most commercially available systems used in agriculture have no means for an integral control to bring interior conditions to set point values; an exception is for some greenhouse systems which 'drive to set point' by various ad hoc techniques. Significant research to improve building heating and ventilation control systems can be found in the literature (e.g. Berckmans and Goedseels, 1986; Timmons and Gates, 1987; Allison et al., 1991; Zhang et al., 1993a,b; Chao et al., 1995; Timmons et al., 1995; Chao and Gates, 1996) .. Fuzzy logic offers an alternative to conventional stage controllers (Gates et al., 1997) . By suitable selection of input/output linguistic variables and a rule base, a broad range of desirable control outcomes can be achieved. Possible features might include user-specified overall control 'tightness' analogous to a control range, closer adherence to set point conditions if desired, and the ability to explicitly set the trade-off between energy costs and interior environment.. A fuzzy inference system (FIS) approach for design of a ventilation control system is developed in this paper. Appropriate design parameters are identified for different degrees of sophistication. Representative systems are simulated and comparisons are made to conventional staged control (CSC) of ventilation. A broiler house model was selected as a candidate test model because both internal heat loads and external disturbances can range over broad degrees, and thus severely test the adequacy of any control system to balance control precision, energy use, and control stability. The proposed fuzzy logic control technique has the following advantages: 1. Simple user inputs to control system. 2. User has supervisory control without worrying about implementation details. 3. Control system is scaleable over range of building sizes. 4. System provides trade-off between departure from set point and energy use. 5. Building ventilation system configuration with controller is flexible (no stage differentials, hysteresis, etc.). 6. Straightforward modification of existing microprocessor-based conventional stage control systems having floating point capabilities. A summary of the three controllers' performance statistics for the simulated 24 h diurnal cycle are presented in Table 5 . The simple one input FLC performed quite well compared with the CSC and with the 3-Input FLC. No control system could reduce maximum control error at high internal heat load as outside temperature exceeded inside set point. The trend for the CSC, as internal heat load increased from 0 to 300 kW, was an increased RMS error, and more extreme values of control error. RMS error increased in the 3-Input FLC as the Energy Use input was reduced, with a setting of about 0.5 yielding similar performance to that of the CSC at the same internal heat load.. The concept of the 3-input FLC is straightforward to implement in any CSC utilizing a microprocessor with floating point capability, since the design concept incorporates the discrete nature of existing staged ventilation control systems. Adoption of such a technique has several advantages. A principal advantage is to provide the building operator with a preferential input to balance energy use and control precision. Conventional staged controllers provide this to a limited extent only, by using variable differential temperatures between stages, and cannot approach a zero steady-state control error because there is no means for implementing an integrator into the CSC. Conventional PI controllers are capable of removing steady-state error, but require tuning for each type of facility, and re-tuning if equipment selection changes in a given facility. Further, PI controllers cannot be readily adjusted to balance energy use. In contrast, the FLC designs presented here can be readily incorporated into existing facilities, and the logic is scaleable over a variety of building and equipment sizes. The FLC appears to retain the robustness and flexibility of the CSC, with enhanced control features including recovery from step and diurnal disturbances.

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O