Interactive Image Data Labeling Using Self-Organizing Maps in an Augmented Reality Scenario Abstract-We present an approach for the convenient labeling of image patches gathered from an unrestricted environment. The system is employed for a mobile Augmented Reality (AR) gear: While the user walks around with the head-mounted AR-gear, context-free modules for focus-of-attention permanently sample the most "interesting" image patches. After this acquisition phase, a Self-Organizing Map (SOM) is trained on the complete set of patches, using combinations of MPEG-7 features as a data representation. The SOM allows visualization of the sampled patches and an easy manual sorting into categories. With very little effort, the user can compose a training set for a classifier, thus, unknown objects can be made known to the system. We evaluate the system for COIL-imagery and demonstrate that a user can reach satisfying categorization within few steps, even for image data sampled from walking in an office environment. The increase of computational power accompanied by ever continuing miniaturization has made mobile systems become a part of the everyday life. One of the main challenges in mobile computing is the development of user assistance systems which can support a human in variable and "generic" tasks. Examples for such tasks in everyday life are the memorization of faces or the retrieval of lost objects. In this paper, we describe a mobile system which is capable of interactive object learning from examples and retrieving objects out of view. The focus of this contribution is on the efficient organization and visualization of the online labeling process using the mobile gear by means of self-organizing maps (SOMs) [Kohonen, 1982] , [Kohonen, 1995] .. The mobile system used here is an augmented reality (AR) gear with two head-mounted cameras and a display by which the user views the original scene together with overlays ("augmentations") of the system (Fig. 1) . Our system employs the view-based approach for object recognition and interactive object learning.. The concept of memorizing views instead of explicit geometric and surface reflectance models offers the well-known advantage that object knowledge can be conveniently acquired from samples (e.g. [Koenderink and van Doorn, 1979] , [Murase and Nayar, 1995] , [Mel, 1997] ). However, still sample views must (i) be labeled, (i.e. the object class must by supplied by the user in some way), and ( ii) be made available in sufficient number to cover the complete "appearance space" of the object. While this problem applies to view based recognition in general, it imposes particular difficulties to mobile systems, where labeled view acquisition requires user effort.. In earlier versions of the AR-system presented here, the user had to present the objects to the system one by one and isolated on homogeneous background. Using the mobile interface, typically 5-10 sample views were acquired per object quite conveniently, then the classifier was re-trained internally [Heidemann et al., 2004a] , [Heidemann et al., 2004b] . However, this procedure still requires considerable effort and is, moreover, unsatisfying from a cognitive point of view: Children do not learn to recognize an object in a one-time process of memorizing views under a given class label! Instead, children explore their surroundings by repeated interaction with many objects and often learn the label (i.e. the class) of an object much later. I.e., they form the concept of a certain object much earlier than learning its name.. Led by these considerations, the concept of the mobile object recognition system presented here can be summarized as follows: The system samples views from its environment permanently while the user moves around in a restricted environment (e.g. an office). Since it is impossible to memorize the entire input stream of images, context-free modules for focusof-attention provide a selection of possibly "interesting" image patches, a subset of which hopefully contains the relevant objects. After a period of sampling, the user can switch into "labeling" mode. The system then trains a SOM using features extracted from the sample patches and visualizes prototypes of the samples in the spatial organization imposed by the SOM. Thus, the system provides a "pre-categorization" of the viewed domain, which allows the user to select and label the subset of relevant objects (e.g. cup, pencil) while discarding others (e.g. background structures).. SOMs are particularly well-suited for the combined task of (i) mapping a high-dimensional and highly non-linear data distribution to only two dimensions while (ii) conserving local neighborhood relations for fast and easy-to-use visualization, see e.g. [Kohonen, 1995] , [Martinetz and Schulten, 1994] , [Villmann et al., 2003 ]. There are numerous applications of SOMs for unsupervised clustering and visualization, examples can be found in [Somervuo and Kohonen, 1999] or [Tokutaka et al., 1999] . Most similar in thought to the present application is probably the PICSOM system, which organizes images using tree-structured SOMs to support the user in retrieval tasks [Laaksonen et al., 2002] . An overview of the multifaceted applications of SOMs is given in [Oja et al., 1999] .. However, pre-categorization achieved by the SOM can only be as good as the underlying data representation. Therefore, features for the characterization of the extracted image patches have to be found which not only reduce data dimensionality but also generate descriptions of high variance, a well-balanced cluster structure and high discriminative power. Only if feature extraction fulfills these criteria, is there a chance to obtain semantically meaningful categories by clustering techniques. In the context of compression, refs. [Manjunath et al., 2001] and [Sikora, 2001] propose MPEG-7 features to encode efficiently the majority of image characteristics. Since it is well known that combining different types of features helps to fulfill the above criteria [Deselaers et al., 2004] , the combinations of MPEG-7 features have been analyzed statistically in [Eidenberger, 2003] . In this work, SOMs are used to analyze contributions of different MPEG-7 features to variance and cluster formation. Because of the good results MPEG-7 features have achieved, we employ part of the feature set in combination with a SOM for fast clustering, visualization and interactive labeling in the mobile system.. After this short outline of the idea, we will describe first the scenario and the interactive functionality of the system in section II, then the features sets (section V) and the iterative SOM training (section III). The system is evaluated in section VI on the COIL-dataset, an example for the efficiency achieved by iterative SOM projections is given in a "real world" scenario in section VII. Results and future lines of development are discussed in section VIII.

O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Automatic Assessment of 3D Modeling Exams Abstract-Computer-based assessment of exams provides teachers and students with two main benefits: fairness and effectiveness in the evaluation process. This paper proposes a fully automatic evaluation tool for the Graphic and Virtual Design (GVD) curriculum at the First School of Architecture of the Politecnico di Torino, Italy. In particular, the tool is designed for the 3D modeling course, taught during the second year, where students have to prove their ability to model static scenes using the open source modeler Blender. During the final exam, students are required to create a 3D model as similar as possible to a reference object proposed by the teacher and shown through a set of 2D views; the similarity of the images is judged according to both model shape and materials. The traditional assessment process is particularly slow and strongly based on teachers subjective evaluation; the proposed solution efficiently implements an objective assessment mechanism that exploits computer vision and image analysis algorithms to automatically extract similarity indices. These indices are related to partial evaluation grades, which are then combined to obtain the final mark. A comparison with the traditional assessment process shows robustness and trustworthiness of the designed approach. T HE Graphic and Virtual Design (GVD) curriculum at the First School of Architecture of the Politecnico di Torino, Italy (three years, roughly corresponding to the BS in the American System) aims to train students in disciplines concerning the design and implementation of graphic and multimedia content. Students are taught the fundamentals of art history and communication techniques as well as the fundamentals of computer science, computer graphics, and multimedia.. In particular, three courses are focused on topics in computer graphics: the first course aims to teach the fundamentals of computer graphics, and it functions as an introduction to 2D concepts, the second course presents methodologies and tools for modeling 3D static scenes, while the last course is devoted to teaching basic techniques for animation and interactive graphics.. The 3D modeling course considered in the present work counts for four credits in the European ECTS system [1] , which are equivalent to 60 hours of teaching. Given the current schedule of the School, the courses have severe time constraints; in particular, the considered course fits into a 12-week slot and is structured in lessons of 5 hours each. Each lesson is taught in a laboratory and organized as follows: the teacher introduces the topic of the lesson and presents the basic theoretical concepts; for instance, when explaining concepts related to solid modeling, Constructive Solid Geometry (CSG) rules [2] are illustrated. Afterward, students work through one or more tutorials step-by-step using the open source modeling tool Blender [3] ; finally, the teacher asks the students to build a model starting from a set of screenshots (2D rendered views of a 3D model) and individually supports the students in completing their task.. The last phase of each lesson allows students to simulate the final exam; the goal of the exam is to evaluate the students ability to create a 3D model as similar as possible to a reference model provided by the teacher and represented through a set of screenshots. Students can choose, in general, among three to five different models (some examples are shown in Fig. 1) ; a target number of polygons is specified for each model (low-poly modeling is a worthwhile task), and images for texture mapping are provided, when necessary. Students are asked to create a model as similar as possible to the teachers reference in terms of geometry, materials, and number of polygons. At the end of the exam, each student delivers his or her own Blender project, together with a set of well-defined renderings of the generated model.. An academic board, usually comprised of three teachers, assesses the students work. Each teacher examines the model and the renderings and proposes a mark. The final mark is obtained after a discussion of individual judgments. Unfortunately, this approach has two main drawbacks: the assessment process can be very time consuming and, most importantly, grading is based on a similarity evaluation that is often strongly subjective.. The current work aims to tackle the above issues by proposing a fully automatic assessment tool able to efficiently provide teachers and students with objective evaluations of works of 3D modeling. The results of previous examination sessions using the traditional evaluation method were used to infer objective assessment criteria for use in fine tuning the tool. A comparison between evaluations by the academic board and results generated by the automatic tool is also provided.. The organization of the paper is as follows: Section 2 summarizes the local context and educational goals for the considered course and presents the motivations for the development of an automatic evaluation tool. Related work is discussed in Section 3, and the proposed assessment solution is illustrated in Section 4. Finally, results and remarks can be found in Section 5. This paper presents a software tool supporting the automatic evaluation of final exams of a 3D modeling course within the GVD curriculum at the First School of Architecture of the Politecnico di Torino, Italy. Computer vision and image analysis techniques are exploited to develop a fair and efficient evaluation strategy based on shape and material similarity criteria. Exam sessions of the past academic year have been used to infer objective assessment rules and to fine tune the proposed grading mechanism. Currently, the system is under validation, and it helps the academic board to smooth out the potential subjectivity of the results.. Experimental tests proved the trustworthiness and the efficiency of the proposed solution (evaluation of an exam requires a few seconds), which will be gradually used to reduce the burden associated with manual assessment. When the system has been completely tested on a significant number of exams (a test is planned for all exam sessions of the current academic year), a web-based version will be made available for students' self-assessment and exam training.. Students seem to appreciate this new approach: they know that teachers are now supported by an automatic assessment system, but they do not see the two marks. Students feel the tool, at this stage, improves the assessments made by the academic board, and the number of students asking for explanations after the publication of exam results has been significantly reduced. An example of automatic assessment is illustrated in Fig. 5 . Fig. 5a shows a reference screenshot, Fig. 5b illustrates the corresponding rendering by a particular student, and Fig. 5c reports the silhouette of the model where the pixel coordinates used to check material similarity have been emphasized (in this case, just one point of view has been considered as sufficient to evaluate material similarity; more complex objects would require, in general, a comparison of screenshots generated from all the N cameras). The number of polygons NP used for modeling the spray bottle was 4,861, the same order of magnitude of NP ref , thus leading to M 1 ¼ 0. The dissimilarity index DI was 2,075, thus producing M 2 ¼ À3 (the two models are slightly different in the upper region). The average euclidean color distance ED for the considered pixels was equal to 79, thus leading to M 3 ¼ 0 (the difference is mainly due to the different color saturation of the bottle). The final score was 30/30.. The proposed tool, implemented as an MS Windows application, is currently under validation: for each exam session, the academic board evaluates students' work in the traditional way and compares their marks with the results generated by the designed tool. This kind of comparison is necessary in order to evaluate the trustworthiness of the automatic process. As an example, Table 1 reports a comparison between the proposed tool and the traditional assessment process concerning several students enrolled into an exam session of the current academic year. Two models were considered: the spray bottle in Fig. 2 and the lounge chair in Fig. 4 . Some screenshots delivered by students numbered 6-9 are shown in Fig. 6 .. The mark determined by the academic board is sometimes expressed as a range, because it encompasses different judgments by the three members. It is noticeable that, in general, automatic and traditional evaluations may differ by one or two units; however, for student no. 1, the two marks were very different. In this case, the academic board re-evaluated the exam. It was found that teachers' evaluation had been strongly affected by the main body of the bottle: the student had not set a smooth rendering of the surface. It was a silly mistake that strongly impacted the overall visual appearance, but one that is not related to the ability of the student to model 3D objects. Therefore, the academic board decided to reconsider its evaluation and a 24/30 mark was assigned: an intermediate value between the initial mark and the automatic result.. It is worth observing that with the proposed software, like with automatic assessment tools in general, students may be drilled to focus on certain aspects which are checked through specific criteria hardwired into and mechanically checked by the system, whereas other aspects which a human grader would recognize at once may go undetected. As a matter of example, a model with a circular symmetry may be created, as any other model, by designing the polygonal mesh vertex by vertex. However, a more effective approach would require the application of a spin operation on a specific profile. Similarly, the effect of a rough surface could be either obtained by an image texture, by a procedural bump mapping technique or by a true mesh deformation, and the choice of the best approach would depend on the particular modeling scenario being considered.. However, the designed tool is not able (actually, it is not meant to) evaluate the adequacy of the methodology used to reach a given modeling result, but, rather, it focuses on the result itself. In other words, given the fact that model representations are actually compared, two very similar objects, obtained using different modeling techniques, would be graded in the same way (and this may be undesirable in some cases). Nonetheless, the designed implementation has been developed to mimic as far as possible the traditional assessment based on visual inspection, i.e., to intentionally focus on the result rather than on the process. Hence, in the proposed similarity-based approach, the risk of "influencing" student's behavior is partially mitigated.. Nevertheless, should the evaluation strategy require to consider also methodology-based assessment criteria, ad hoc routines (like those developed in [36] ) could be easily integrated. In this way, both process and result would be considered, making the resulting assessment technique suitable also for other disciplines, like 2D graphics and 2D/3D animation, among others. Computing systems have been used in education since their earliest appearance. However, it was only in more recent years, with the widespread diffusion of personal computers and the evolution of network infrastructures, that the use of the associated technologies significantly impacted all the facets of teaching and learning processes. In the context of learning assessment, software tools started to be developed for the evaluation of test-based assignments and were applied in a variety of educational contexts [5] , [6] , [7] . Various technological solutions were progressively exploited to increase assessment effectiveness [8] , mainly by focusing on strategies for the automatic construction of test sheets [9] , [10] , [11] , [12] .. Despite guarantees in terms of objectivity, when the evaluation of learning outcomes requires considerations of applying knowledge in a specific context, test-based assignments are generally replaced by assessment techniques based on performance evaluation [8] , [13] . The potential for unfair assessment for this type of assignment is higher than for test-based evaluations. As stated in [14] , the main issue with computer-based performance assessment is related to the complexity of translating very specific grading methodologies relying on measurable objectives (e.g., rubrics) into an automatic evaluation logic.. Thus, vertical solutions, each tackling a particular assessment problem, have been proposed. Although a few solutions aimed at computer-based assessment outside scientific domains exist, e.g., automatic essay or pronunciation evaluation [15] , [16] , [17] , most of the works reported in the literature focus on the assessment of technical subjects. Indeed, most of the researchers' attention has been devoted to the automatic assessment and grading of computer programming assignments. Solutions proposed in this field differ in the set of programming languages addressed, in the strategy used for weighting error severity, in the ability to evaluate code quality, in the degree of integration with existing development environments, and in the availability of antiplagiarism features [18] , [19] .. Nonetheless, many fields other than computer science have been explored. For instance, in [20] , a tool for assessing automata-based assignments is proposed. Automatic assessment of formal specification coursework is addressed in [21] . Computer-based grading in the field of database design has been studied in [22] and [23] . A software tool allowing an automatic check and verification of a student's laboratory work in the design and simulation of digital circuits is illustrated in [24] . In [25] , a system for checking exercises in the field of dynamic geometry systems is illustrated. In [26] , a software platform for self-assessment in the automatic control systems domain is presented. In [27], a computer-based system is used to teach and assess industrial robot path planning and control.. Most of the assessment techniques above use some kind of computer graphics and multimedia techniques to enhance their communication potential [28] . In some cases, distinctive features of computer graphics are even used to teach other subjects, as in the case of [29] , where computer game development is exploited to teach object-oriented programming. Furthermore, ever more powerful applications of computer graphics techniques, like virtual and augmented reality, are being exploited to build effective virtual laboratories and further enhance learning experience through virtual tutors [30] , [31] , [32] , [33] , [34] , [35] . However, despite the pervasiveness of the above techniques in the framework of education, only a few works in the literature have focused on the issue of automatically assessing the outcomes of computer graphics-related courses.. In particular, Jiang et al. [36] present a method for autoevaluating Photoshop images and Flash animations. The system considers the content of the electronic file as well as the log of the student's operating processes, and it extracts technical information such as image size, number of layers, font size, etc. Artistic factors are only marginally taken into account by considering the organization of the work in terms of color and object location. Assessment is performed by executing a matching algorithm based on fuzzy logic against a reference in the form of text.. Some efforts have also been devoted to investigating computer-based assessment in the field of Engineering Drawing (ED) and Computer Aided Design (CAD). In fact, whereas this kind of evaluation used to depend only on the final printouts, today the amount of information generated by existing software tools is impressive, and objective evaluation has become an onerous task. However, ED and CAD learning outcomes are generally assessed according to consolidated and structured criteria [37] . Thus, based on the effective rubrics available [38] , several computer-based grading systems were developed.. In particular, in [39] and [40] , an automatic grading tool for a freshman solid modeling course is presented. The tool is designed as an integrated instrument allowing teaching staff to manage the class (e.g., by supervising student to section allocation, generating web pages for the course syllabus, verifying the status of each student, etc.) as well as providing students with a means to check their grades and interact with the instructor in case of doubts about a given result. The tool is implemented as a collection of software modules working on Excel spreadsheets (storing grades), SolidWorks data (models and engineering drawings submitted by the students to a database manager), and web pages (containing student's information). The system handles different types of assignments (quiz, laboratory recitation, hand sketching, and final problems). Some of the assignments are graded automatically. For instance, for laboratory works the student is provided with a reference print-out and a limited set of very specific grading criteria [41] , e.g., all dimensions change together, all cross sections have the same number of edges, etc. The student's goal is not to produce a nice looking picture, but to create a mechanical drawing respecting rigid constraints. The designed tool works on the file produced by the student and checks whether the dimensions and views are present and whether placement of objects is correct. Despite the promising approach, the authors underlined the fact that the design of a fully automatic grading routine for drawings is a complex task. Thus, some of the assignments, like for instance hand sketches, need to be graded visually: in this case, an image file is created and the instructor adds the grade manually.. In [42] , different grading criteria were defined by referring to the "tracing technique," a common method of assessment in the ED domain. In this technique, the teacher's drawing is traced on the student's drawing, and accuracy is measured to arrive at an initial grade. Specifically, this initial mark is computed by taking into account the following "accuracy elements": accuracy of object type (a comparison of the number of objects of the same type in the student's drawing to the number in the teacher's drawing), the accuracy of object measurement (a comparison of the number of object entities with similar attributes between the drawings of the student and the teacher), and the entity of object attribute (a comparison of the number of object entities with the same attribute in the drawings of the student and the teacher). Unfortunately, even if the student's drawing shares the exact number of accuracy elements with the teacher's drawing, it is not guaranteed that the student's drawing is exactly similar to the teacher's drawing (and vice versa). Thus, to give the final mark, the initial mark is adjusted by means of visual comparison manually performed by the teacher.. The above works confirm the need for automatic tools to provide an efficient and objective assessment in the field of computer graphics. However, the proposed methods cannot be applied for the evaluation of 3D modeling assignments. For instance, despite the availability of the necessary assessment information, the approach in [36] would not be effective in the domain tackled by this paper. In fact, the student could follow multiple routes to reach the same result; thus, the analysis of the modeling process would be of little to no help. Moreover, technical and artistic factors in 3D modeling (e.g., similarity with a template shape, materials usage, etc.) go beyond the strict engineering requirements of ED and CAD considered in [42] ; thus, they could not be judged individually by looking at the content of the electronic file, but would have to be considered as a whole by taking into account the concurrent effect of various factors such as camera position, lights configuration, etc. These various factors would necessarily require working with the resulting image rendering. As anticipated, given the importance (and complexity) of visual inspection, a computer-based assessment technique for 3D modeling should, therefore, automate this step.

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O


The Fine Structure of Galls in Phylogenetic Networks A phylogenetic network is a generalization of a phylogenetic tree, allowing properties that are not tree-like. With the growth of genomic data, much of which does not fit ideal tree models, there is greater need to understand the algorithmics and combinatorics of phylogenetic networks (Posada and Crandall 2001, Schierup and Hein 2000) . Wang et al. (2001) studied the problem of constructing a phylogenetic network for a set of n binary sequences derived from the all-zero ancestral sequence, when each site in the sequence can mutate from zero to one at most once in the network, and recombination between sequences is allowed. They showed that the problem of minimizing the number of recombinations in such networks is NP-hard, but introduced a special case of the problem, i.e., to determine whether the sequences could be derived on a phylogenetic network where the recombination cycles are node-disjoint. Wang et al. (2001) provide a sufficient, but not a necessary test, for such solutions. Gusfield et al. (2003 Gusfield et al. ( , 2004 ) gave a polynomial-time algorithm that is both a necessary and sufficient test. In this paper, we study in much more detail the fine combinatorial structure of node-disjoint cycles in phylogenetic networks, both for purposes of insight into phylogenetic networks and to speed up parts of the previous algorithm. We explicitly characterize all the ways in which mutations can be arranged on a disjoint cycle, and prove a strong necessary condition for a set of mutations to be on a disjoint cycle.The main contribution here is to show how structure in the phylogenetic network is reflected in the structure of an efficiently-computable graph, called the conflict graph. The success of this approach suggests that additional insight into the structure of phylogenetic networks can be obtained by exploring structural properties of the conflict graph.

O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O


Thermocapillary Migration of a Drop: An Exact Solution with Newtonian Interfacial Rheology and Stretching/Shrinkage of Interfacial Area Elements for Small Marangoni Numbers In this paper we analyze the effects of the following phenomena associated with the thermocapillary migration of a drop. The first is the influence of Newtonian surface rheology of the interface and the second is that of the energy changes associated with stretching and shrinkage of the interfacial area elements, when the drop is in motion. The former occurs because of dissipative processes in the interfacial region, such as when surfactant molecules are adsorbed at the interface in sufficient concentration. The interface is typically modeled in this instance by ascribing to it a surface viscosity. This is a different effect from that of interfacial tension gradients arising from surfactant concentration gradients. The stretching and shrinkage of interfacial area elements leads to changes in the internal energy of these elements that affects the transport of energy in the fluids adjoining the interface. When an element on the interface is stretched, its internal energy increases because of the increase in its area. This energy is supplied by the neighboring fluids that are cooled as a consequence. Conversely, when an element on the interface shrinks, the adjoining fluids are warmed. In the case of a moving drop, elements of interfacial area are stretched in the forward half of the drop, and are shrunk in the rear half. Consequently, the temperature variation on the surface of the drop and its migration speed are modified. The analysis of the motion of a drop including these effects was first performed by LeVan in 1981, in the limit when convective transport of momentum and energy are negligible. We extend the analysis of LeVan to include the convective transport of momentum by demonstrating that an exact solution of the momentum equation is obtained for an arbitrary value of the Reynolds number. This solution is then used to calculate the slightly deformed shape of the drop from a sphere. K EYWORDS : thermocapillary migration; Newtonian interfacial rheology; small Marangoni numbers; drop; stretching and shrinking of interface The thermocapillary migration of drops was first investigated theoretically by Young, Goldstein, and Block. 1 When a drop is present in an unbounded liquid that has a uniform temperature gradient, a thermocapillary stress is generated at the interface arising from the variation of interfacial tension with temperature. Typically, the drop is propelled toward the warm portion of the continuous phase. Young et al. obtained a result for the steady velocity of a drop, including a contribution from the effect of gravity, in the limit when the Reynolds and Marangoni numbers approach zero, so that the effect of convective transport of momentum and energy can be neglected. To verify their theory, Young et al. performed experiments on air bubbles in silicone oils, in which the effects of thermocapillarity and gravity were counterbalanced, so that the bubbles were nearly stationary.. Balasubramaniam and Chai 2 showed that the solution obtained by Young et al . is an exact solution of the Navier-Stokes equations for any value of the Reynolds number, when the Marangoni number is equal to zero and the effect of gravity is not present. A discussion of other pertinent literature on the motion of drops due to thermocapillarity can be found in Subramanian and Balasubramaniam. 3 Our objective in this paper is to consider the effect of Newtonian interfacial rhelogy and stretching/shrinkage of interfacial area elements on the thermocapillary motion of a drop. In the limit when the Reynolds and Marangoni numbers approach zero, this problem has been addressed by LeVan 4 and discussed by Subramanian and Balasubramaniam. 3 The rheology of the surface needs to be considered in special situations, for example, when surfactant molecules are adsorbed on the interface in sufficient quantity so that an interfacial viscosity can be ascribed to the interface. As a consequence the drop encounters additional resistance to its motion. Scriven 5 has provided the framework for a Newtonian model of the interface, in which the contribution to the surface stress depends linearly on the rate of deformation of area elements on the interface. A detailed discussion of surface rheology models is available in Edwards, Brenner, and Wasan. 6 The second effect that we consider is the consequence of the deformation of interfacial area elements on the energy balance at the interface. Even when a drop moves in a continuous phase fluid that is isothermal, temperature variations can be generated on the interface. This is because elements of surface area grow as they move from the stagnation point to the equatorial region in the forward half of the drop. The increase in area increases the interfacial internal energy, which must be provided by the fluids adjoining the interface. Thus, the neighboring fluids are cooled. In the rear portion of the drop, the situation is opposite, and the adjoining fluids are warmed. Therefore, an interfacial temperature variation can occur, and the consequent thermocapillary stress influences the motion of the drop. This effect has been considered by Harper et al. , 7 Kenning, 8 LeVan, 4 and Torres and Herbolzheimer. 9 One infers from the analysis presented by Subramanian and Balasubramaniam 3 that the thermocapilliary migration velocity of the drop is unaffected by the surface shear viscosity, but is influenced by the surface dilatational viscosity and the effect of stretching and shrinkage of interfacial area elements. The velocity of the drop is reduced as a consequence. We shall revisit the analysis performed by LeVan 4 and Subramanian and Balasubramaniam 3 below and show that their results are valid even when the convective transport of momentum is included in the analysis. The effect of inertia is shown to alter the pressure field, and lead to deformation of the drop. The small change in the shape of the drop from a sphere is obtained using a technique similar to that used by Balasubramaniam and Chai. 2 We have shown that a solution in the limit of zero Reynolds and Marangoni numbers for the velocity and temperature fields, obtained elsewhere 4, 3 for the thermocapillary migration of a drop accommodating Newtonian surface rheology and the influence of internal energy changes associated with the stretching and shrinkage of surface elements, holds for arbitrary values of the Reynolds number so long as the Marangoni number is set equal to zero. The pressure fields are modified by inertia. The leading order result for the migration speed of the drop is the same as that obtained when the Reynolds number is zero. Furthermore, whereas the drop will be

B I O O B O O O O B I I O O O O O O O B I I O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I B I I B I I B B I I I I O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O


Structure Tensor Based Image Interpolation Method Abstract -Feature preserving image interpolation is an active area in image processing field. In this paper a new direct edge directed image super-resolution algorithm based on structure tensors is proposed. Using an isotropic Gaussian filter, the structure tensor at each pixel of the input image is computed and the pixels are classified to three distinct classes; uniform region, corners and edges, according to the eigenvalues of the structure tensor. Due to application of the isotropic Gaussian filter, the classification is robust to noise presented in image. Based on the tangent eigenvector of the structure tensor, the edge direction is determined and used for interpolation along the edges. In comparison to some previous edge directed image interpolation methods, the proposed method achieves higher quality in both subjective and objective aspects. Also the proposed method outperforms previous methods in case of noisy and JPEG compressed images. Furthermore, without the need for optimization in the process, the algorithm can achieve higher speed 1 .. Feature preserving image interpolation is an active area in the image processing field, from everyday digital pictures to application-oriented medical and satellite images. Many methods have been proposed in the past decades to tackle this problem [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] . Generally speaking, the methods for image interpolation/super-resolution can be divided in 3 different categories: 1) Direct Interpolation methods, 2) PDE based interpolation methods and 3) Optimization based interpolation methods. All of these methods have their pros and cons regarding their simplicity of implementation, computational complexity and performance. The proposed method in this paper is a direct interpolation method without the need for any optimization in the process. Also in terms of computational time, the proposed method can achieve the result in less than one second for an image of typical size using MEX based implementation. This feature along with not using any optimization procedure, as well as being robust in case of noisy images, make this method a suitable choice for implementation in everyday used electronic devices.. Nearest neighbor and bilinear interpolation are two simple methods for image interpolation [1] . Despite the simplicity in implementation and very low computational cost, these methods suffer from severe blocky artifacts, as well as blurring and ringing artifacts near the edges. Although better performance can be achieved by using higher order splines, rather than 0 and 1 order splines as in the nearest neighbor and bilinear methods, higher order spline methods still contain oscillatory edges and ringing artifacts [2] . The main reason is that these methods don't take into consideration any information other than intensity values. In other words, they are intensity based and not feature (edge) based. So even though they are easy to implement and need low computational cost, they are not suitable for most of applications.. The final recipient of any image processing algorithm is the human visual system which is very feature sensitive. These features are mostly edges and corners within the image. Also sharpness of the final image is of high importance. Based on these criteria, the previously mentioned methods, despite their technical advantages, are not satisfactory. So the need for introducing new approaches and novel models for image interpolation which satisfy the human visual system has been emerged in the past decades and many methods have been proposed. Some of these methods will be mentioned here. Edge directed methods usually are the first ones that come to notice when dealing with image interpolation problem. In 2001 a method called NEDI was proposed which performs based on the assumption that every image can be modeled as a locally stationary Gaussian process [3] . Based on this assumption, the local covariance coefficients from the low resolution (LR) image is estimated and then interpolation is done based on the geometric duality between the LR covariance and the high resolution (HR) covariance. An improved version of NEDI algorithm called iNEDI is proposed later which achieves higher scores in terms of subjective and objective image quality measures relative to NEDI with the cost of needing more computational time [4] . Another edge directed image interpolation method is ICBI which works based on an estimation of the edge orientation using second order derivative of the image [5] . DFDF method [6] is another method in this category which utilizes directional filtering and data fusion. In DFDF, at first, two observation sets are defined in two orthogonal directions for each pixel to be interpolated. Then these two estimates will be fused using Linear Minimum Mean Square Error (LMMSE) in order to achieve a more robust estimate for the missing pixel.. Methods proposed in [7] [8] [9] [10] [11] [12] [13] [14] [15] also are good examples of edge directed image interpolation. In [7] , the method is based on partitioning the input image into homogeneous and edge areas with regard to local structure of the image and then, interpolating each parts differently, bilinear interpolation for homogeneous regions and an adaptive edge oriented method for edge pixels. In [8] , a modified edge adaptive bilinear image interpolation method called EASE is proposed. This modified version is achieved using the classical interpolation error theorem. In [9] , a new directional cubic convolution (CC) interpolation scheme is proposed. In [10] , an interpolation framework is proposed in which denoising and image sharpening are embedded together. In this method, bilateral filtering method is used to partition the input image into detail and base layers, and then edge preserving interpolation method is applied to each layer. In [11] , the edge information of the LR image is first estimated using the modified Leung-Malik filter bank, and then this information is converted into that of HR image by using a mapping function. In [12] , a fast image interpolation method with adaptive weights is proposed motivated by Inverse Distance Weighting (IDW). The use of Radial Basis Functions (RBFs) to solve image interpolation problem is investigated in [13, 14] . In [15] , a soft decision interpolation technique is proposed which estimates missing pixels in groups rather than one at a time. They use a piecewise 2-D autoregressive (AR) model to determine the local structure of the scene.. Even though the above mentioned methods are of a wide range of use and discipline, still there are more methods that are not discussed here; like Partial Differential Equation (PDE) based methods [16, 17, 27] , and regularization based methods [18] [19] [20] . The reader will be referred to the papers and their references for more information on these classes of image interpolation methods.. As can be seen, each of the mentioned methods deals with the image interpolation problem from a different angle. But still image interpolation is an open problem and there is room for improvement. In this paper, a new edge-directed method based on structure tensor will be proposed which its strength is not only in reconstructing edges in the HR image, but also is more robust in case of noise. The proposed method is very simple and easy to implement and based on the conducted experiments, outperforms the most common image interpolation methods. For comparison, five well-known image interpolation methods are considered: NEDI [3] , DFDF [5] , ICBI [6] , KR [26] and iNEDI [4] . Tests were conducted for noise-free, noisy and JPEG compressed images. For completeness of the comparison another structure tensor-based method by Roussos and Maragos [27] is also considered. This method (RM) can be categorized as a PDE-based technique. In this method at first an initial interpolation is done by Fourier zero-padding and de-convolution. The result of this stage suffers from significant ringing artifacts. Using a tensor-driven diffusion process, the ringing artifacts are removed. The main assumption in this method is that the process of interpolation is a reversible process which cannot be hold always. Based on this assumption, interpolation is done by first applying an anti-aliasing low-pass filter followed by sampling. This assumption can be problematic especially in the case of naïve sub-sampling which is the case used in this paper. In naïve sub-sampling of factor ܰ, one pixel is chosen from each ܰ pixels of the image without any anti-aliasing filtering. This will cause high amount of ringing artifacts near edges introduced by the first stage of the method as well as sever stair-cased edges which cannot be resolved properly using the tensor-driven diffusion process. More discussion will be given in the following sections regarding this issue. Here the online implementation of this method is used implemented by Getreuer [28] .. The rest of the paper is organized as follows: in Section 2 a brief introduction will be given on structure tensor computation and its theoretical aspects. Then in Section 3, the proposed interpolation method will be described in more detail. Section 4 contains the implementation aspects, image quality measures that being used and tables of objective and subjective comparison between the five above mentioned methods and the proposed method, as well as some of the final results. Section 5 concludes the paper. In this paper, a new edge directed method for image interpolation based on structure is proposed which takes into account the advantages of structure tensor to determine the edge orientation as well as corner points of an image. Even though the concept of structure tensor is the same as gradient vectors, due to the presence of noise and discontinuity of edges caused by image compression, downsampling and digitizing, the structure tensor provides more robust edge orientations. Also using structure tensor, corner points can be better distinguished from other features such as edges.. The proposed method is tested for noise-free, noisy and JPEG compressed images. Numerous comparisons were made against several popular image interpolation methods. In most cases the proposed method outperformed the other methods both subjectively and objectively, especially in case of noisy and compressed images with noticeable margin. Also in terms of computational time, the proposed method can achieve the result in less than one second for an image of typical size using MEX based implementation. This feature along with not using any optimization procedure, as well as being robust in case of noisy images, make this method a suitable choice for implementation in everyday used electronic devices; Not to forget parallelizability using GPU based implementations. Apparently a straightforward implementation of the proposed algorithm can be very time consuming. For example, assuming a 5x5 neighborhood size for each new pixel to be interpolated, 25 weights for each of the distance based and structure tensor based weights should be computed before the computation of the weighted summation. Fortunately, in most digitized images, only a small portion of pixels are edge/corner regions leaving a large number of pixels in uniform regions with very small variations in gray values. These regions don't contain as much important information as edges/corners and hence they can be easily and efficiently interpolated using simple and fast interpolation methods like bilinear interpolation. To do this, a very simple pair of gradient masks is implemented. Fig.3 shows the gradient masks: Convolving these masks with the input image, we can compute the gradient values for each pixel in horizontal and vertical directions and the magnitude of the gradient is computed using = ඥ(݃ ௫ ଶ + ݃ ௬ ଶ ) for each pixel. After normalizing the magnitude into the range [0,100], a simple threshold (T) is applied in order to filter out the pixels in relatively uniform regions, where the pixels will be interpolated with a simple and fast interpolation method. Another issue rises in dealing with corner points. When computing the structure tensor, the Gaussian filter tends to smooth and round the corners. To that end, the corner points should be treated differently than edge points. Using the structure tensor, identification of corner points is easy: not only the smaller eigenvalue of the structure tensor significantly is bigger than 0, but also the ratio of the smaller and bigger eigenvalues is greater than that for edge pixels. Using this criterion, corner points can be detected.. Based on the above considerations, the proposed algorithm is given below.

O O O B I O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On the mathematical synthesis of equational logics Abstract. We provide a mathematical theory and methodology for synthesising equational logics from algebraic metatheories. We illustrate our methodology by means of two applications: a rational reconstruction of Birkhoff's Equational Logic and a new equational logic for reasoning about algebraic structure with name-binding operators. We have introduced a categorical framework for the synthesis of equational logics. This comprises a general abstract notion of equational presentation together with an equational deduction system that is sound for a canonical model theory. In this context, we have also introduced a mathematical methodology for establishing completeness. This is based on an internal strong completeness result that typically leads, through an analysis of the construction of free algebras, to a characterisation of satisfiability via a rewriting-style deduction system embedded within the equational deduction system. Two applications of our theory and methodology were presented. They respectively provide a rational reconstruction of Birkhoff's Equational Logic and a novel nominal logic for reasoning about algebraic structure with name-binding operators. A further major application was given in Fiore and Hur (2010) with the synthesis of Second-Order Equational Logic: a deductive system for equational reasoning about languages with variable binding and parameterised metavariables (see also Fiore and Mahmoud (2010) ).. The extension of the theory of this paper from the mono-sorted to the multi-sorted setting requires a more involved categorical theory (see Fiore (2008) ; Fiore and Hur (2008); Hur (2010)) . A yet more comprehensive extension for a theory of rewriting modulo equations has also been developed. Lemma B.2. For S = (C , T, A) a MES admitting free algebras, the free S-algebra monad T S on C is strong. The components of the strength st S are given by the unique maps such that the following diagram commutes: The naturality of st S follows from the fact that, for all f : V / / V ′ and g : C / / C ′ , the maps. are both an homomorphic extension of η S V ′ ⊗X ′ • (f ⊗ g) : X / / [V, T S (V ′ ⊗ X ′ )] with respect to the S-algebra [V,. Three of the four coherence conditions for strength follow from the fact that the maps Lemma B.3. Let S = (C , T, A) be a MES admitting free algebras. Then, the quotient natural transformation q S : T . / / T S is a strong functor morphism between the strong monads T and T S . That is, the following diagram commutes:. Proof. The commutativity of the diagram follows from the fact that both We are now ready to prove the internal completeness theorem.. Proof of Theorem 3.1. We show 1 ⇒ 2 ⇒ 3 ⇒ 1.. 1 ⇒ 2. Holds vacuously. • w) where s * : T S X / / X is the unique homomorphic extension of the identity map on X with respect to the S-algebra (X, s).

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Robust scheduling and robustness measures for the discrete time/cost trade-off problem Projects are often subject to various sources of uncertainties that have a negative impact on activity durations and costs. Therefore, it is crucial to develop effective approaches to generate robust project schedules that are less vulnerable to disruptions caused by uncontrollable factors. In this paper, we investigate the robust discrete time/cost trade-off problem, which is a multi-mode project scheduling problem with important practical relevance. We introduce surrogate measures that aim at providing an accurate estimate of the schedule robustness. The pertinence of each proposed measure is assessed through computational experiments. Using the insights revealed by the computational study, we propose a two-stage robust scheduling algorithm. Finally, we provide evidence that the proposed approach can be extended to solve a complex robust problem with tardiness penalties and earliness revenues. In project management, it is often possible to expedite the duration of some activities and therefore reduce the project duration with additional costs. This time/cost trade-off has been widely studied in the literature focusing on linear and continuous time/cost relationships. In this paper, we address the discrete version, namely the discrete time/cost trade-off problem (DTCTP), which is a multi-mode project scheduling problem having practical relevance. Project managers often allocate more resources to accelerate the activities and each resource allocation defines an execution mode. Thus, multiple alternatives usually exist to execute an activity. DTCTP utilizes only one single nonrenewable resource (money) and does not explicitly consider renewable resources (e.g. machines, equipment and staff), which are available at constant amounts in every instance of the planning period.. Formally, the DTCTP is defined as follows. Given a project with a set of n activities along with a corresponding precedence graph G = (N, A), where N is the set of nodes that refer to the activities of the project, and A & N Â N is the set of immediate precedence constraints on the activities. It is noteworthy that G also includes two dummy ''start" and ''end" nodes indexed by 0 and n + 1, respectively. Each activity j (j = 1,. . ., n) can be performed at one of the modes chosen from the set M j . Each mode m 2 M j , is characterized by a processing time p jm and a cost c jm .. Two basic versions of the DTCTP have been defined in the literature so far: the deadline problem (DTCTP-D) and the budget problem (DTCTP-B). In the deadline problem, given a project deadline d, one of the possible modes is assigned to each activity so that the makespan does not exceed d and the total cost is minimized. The budget problem, on the contrary, minimizes the makespan while not exceeding a maximum preset budget B. Despite its practical relevance, the research on DTCTP is rather sparse due to its inherent computational complexity (it has been shown to be strongly NP-hard for general activity networks (De et al., 1997) ). In their comprehensive review papers, De et al. (1995) and Weglarz et al. (2010) discuss the problem characteristics as well as exact and approximate solution strategies. We refer the readers to the papers of Demeulemeester et al. (1996 Demeulemeester et al. ( , 1998 for exact algorithms and to Skutella (1998) , Akkan et al. (2005) , Vanhoucke and Debels (2007) and Hafızoglu and Azizoglu (2010) for approximate algorithms. Furthermore, Erengüç et al. (1993) apply Benders decomposition to solve the time/cost trade-off problem with discounted cash flows, which combines the DTCTP and the payment-scheduling problem.. 0377-2217/$ -see front matter Ó 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2010.05.046. The existing studies on DTCTP generally assume complete information and deterministic environment. However, in practice, projects are often subject to various sources of uncertainty that may arise from the work content, resource availabilities, project network, etc. A schedule that is optimal with respect to project duration or cost may largely be affected by these disruptions. Therefore, it is crucial to develop effective approaches to generate project schedules, which are less vulnerable to disruptions caused by these uncontrollable factors. To the best our knowledge, the only paper which addresses uncertainty on DTCTP is by Klerides and Hadjiconstantinou (2010) ; they used stochastic programming to model uncertain activity durations.. The contribution of our paper is threefold. First, we introduce a new version of the DTCTP under uncertainty with tardiness penalties and earliness revenues. Second, we propose some surrogate measures to evaluate schedule robustness. The quality of the proposed schedules is assessed through several performance measures. Finally, we develop a two-phase approach for generating robust schedules. The solution approach integrates an analytical tool to support the decision makers in budget allocation decisions and a robust scheduling algorithm. The developed scheduling algorithm addresses the crucial need to construct robust project schedules that are less vulnerable to disruptions caused by uncontrollable factors. Furthermore, it serves as a basis to develop decision support systems (DSS) to help project managers in planning under uncertain environments. To address the crucial need to build robust project schedules that are less vulnerable to disruptions caused by uncontrollable factors, we have investigated the robust scheduling of a variant of the multi-mode discrete time/cost trade-off project scheduling problem. In this variant, the problem is to select a mode for each activity so that the project is completed within a preset deadline and the total cost is minimized. We describe and analyze the pertinence of several robustness measures. We provide empirical evidence that the project buffer size is the more appropriate robustness measure regardless of the network complexity. Based on this finding, we propose a two-step methodology for generating robust schedules. In the first phase of the methodology, we determine the minimum required budget. Next, in the second phase, this budget is slightly inflated by a specified amplification factor and then the buffer size is maximized. We provide strong empirical evidence that the budget amplification consistently improves the schedule robustness. Therefore, we address the important issue of determining the best trade-off between project cost and schedule robustness. To that aim, we propose an extended model involving both tardiness penalties and earliness revenues and we describe an appropriate solution strategy that requires a restricted number of simulations.. A promising research focus could be deriving robust schedules for the multi-mode resource constrained project scheduling problem. This problem might be viewed as a natural combination of the well-known resource constrained project scheduling problem and the time/cost trade-off problem. We expect that the results presented in this paper could provide a useful base for investigating this challenging problem.

O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O


Locating and Characterizing the Stationary Points of the Extended Rosenbrock Function Two variants of the extended Rosenbrock function are analyzed in order to find the stationary points. The first variant is shown to possess a single stationary point, the global minimum. The second variant has numerous stationary points for high dimensionality. A previously proposed method is shown to be numerically intractable, requiring arbitrary precision computation in many cases to enumerate candidate solutions. Instead, a standard Newtonian method with multi-start is applied to locate stationary points. The relative magnitude of the negative and positive eigenvalues of the Hessian is also computed, in order to characterize the saddle points. For dimensions up to 100, only two local minimizers are found, but many saddle points exist. Two saddle points with a single negative eigenvalue exist for high dimensionality, which may appear as "near" local minima. The remaining saddle points we found have a predictable form, and a method is proposed to estimate their number. Monte Carlo simulation indicates that it is unlikely to escape these saddle points using uniform random search. A standard particle swarm algorithm also struggles to improve upon a saddle point contained within the initial population. Suitable test functions are indispensable in the development of optimization algorithms, since practical optimization problems are frequently computationally expensive. However, the conclusions that can be drawn about the abilities of an algorithm are limited by the knowledge of the challenges that a particular test function poses. Therefore, we analyze the extended Rosenbrock function in this paper, in order to highlight the challenges that this popular test function poses.. Similar to Shang and Qiu (2006) , we analyze the Hessian of the test functions at a stationary point. A stationary point x of a function f (x) is any point where the gradient vector vanishes, that is, ∇f (x) = 0. However, instead of only determining whether or not the Hessian is positive definite, we compute the eigenvalues of the Hessian. At a stationary point, the ith eigenvalue λ i is the curvature of the function in the direction of the associated eigenvectorx i (Himmelblau, 1972) . If all eigenvalues are positive (negative) at a stationary point, the stationary point is a local minimizer (maximizer). If some eigenvalues are positive, and some negative, the stationary point is a saddle point. In some cases, all but one of the eigenvalues are positive. If in addition the magnitude of this single negative eigenvalue is significantly less than the magnitude of the positive eigenvalues, a very limited range of descent directions exist, and optimization algorithms may find such saddle points difficult to escape. One such an example is found by Deb et al. (2002) for the 20D Rosenbrock problem, where the "near minimum" they report is in fact a saddle point with only one negative eigenvalue. One million small random perturbations about this point do not generate any superior solutions (Shang and Qiu, 2006) . However, a small perturbation in the direction of the eigenvector associated with the negative eigenvalue generates a superior solution. This example motivates the characterization of a test function in terms of the presence of saddle points, particularly those saddle points with very few negative eigenvalues. We investigated the stationary points of two variants of the extended Rosenbrock problem. Variant A is shown to have a single stationary point, the well known global minimum. Variant B, however, contains numerous stationary points, including two local minima. The numerical scheme proposed by Shang and Qiu (2006) is shown to be a numerically intractable method to find all stationary points. Instead, Newton's algorithm is employed with multiple random starts. This uncovered sufficient stationary points in order to deduce a pattern in which the stationary points appear. The stationary points are characterized in terms of the number of negative eigenvalues, and the relative magnitude. Few negative eigenvalues of small magnitude as compared to the positive eigenvalues indicates "near" local minima, in that very limited search directions exist that can improve the solution. Variant B contains two such "near" local minimum saddle points, each with a single negative eigenvalue. It also contains numerous others that have a small fraction of negative eigenvalues, again with a small magnitude as compared to the positive eigenvalues. Using Monte Carlo simulation we illustrated that uniform random search is unlikely to escape these saddle points of Variant B. An algorithm like the particle swarm optimization (PSO) for instance struggles to escape a saddle point contained in the initial population. Finally, we found that the PSO algorithm does in fact locate saddle points while searching for the minimum, but manages to escape most of the time.

O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O