Solving an urban waste collection problem using ants heuristics This paper describes the methodology that we have applied for the solution of an urban waste collection problem in the municipality of Sant Boi de Llobregat, within the metropolitan area of Barcelona (Spain). The basic nature of the considered problem is that of a capacitated arc routing problem, although it has several specific characteristics, mainly derived from traffic regulations. We present the model that we have built for the problem, which results after an appropriate transformation of the problem into a node routing one. We also present the ant colonies heuristics that we have used to obtain the solutions to the problem. These combine constructive methods, based on nearest neighbor and on nearest insertion, with a local search that explores various neighborhoods. The application of the proposed methods gives results that improve considerably the ones that were previously used in the municipality. The management of urban waste is now undergoing a strong change and innovation phase as required by the growing environmental concern from citizens and governments. As a result, in recent years there has been an increasing number of directives from the European Union and regulations in the different countries. In particular, in 2000 the National Plan for Urban Waste (PNRU, from its name in Spanish) gave some unifying criteria to be applied in Spain in this field. According to the plan, municipalities with more than 5,000 inhabitants are forced to ensure proper separation for selective collection of garbage. New and interesting management problems arise in the affected municipalities: what collection system should be applied, where to locate the garbage accumulation areas as well as the collection points, how many containers and of what type should be assigned to each area, which are the most appropriate collection routes, which frequency of collection should be applied in each section, how big the fleet of vehicles should be, and so on.. In Catalonia alone, there are 120 municipalities affected by this regulation, 28 of which belong to the metropolitan area of Barcelona. Sant Boi de Llobregat, is one such municipality with more than 80,000 inhabitants. The need to adapt to the new legislation and the desire to renew equipment and to design a new physical collection system led the Council and the municipal transportation company, to sign an agreement with the Technical University of Catalonia in 1999 as described by Bautista in [1] . One result of this collaboration is the decision support system Integrated System of Selective Urban Garbage Collection (SIRUS, from its name in Spanish). SIRUS was designed to help decision-making in problems related to the management of garbage collection, on the basis of a set of conditions and criteria given by the new regulations. This work deals with one of the aspects of the above mentioned project. Namely, the design of collection routes in Sant Boi de Llobregat and the integration of the proposed method within SIRUS.. Similarly to other countries, in Spain waste generated in urban areas is collected by municipal organizations or its collection is run by private companies under some type of contract. Urban waste is collected and transported to its final destination, trying to offer a quality efficient service at minimum cost. Like in other countries of the European Union, in Spain most of the population is concentrated in urban areas with a very high population density, given that people usually live in apartment buildings. As a result, the collection technique has evolved away from doorto-door collection, very common in the USA, and the predominant system consists of collection points located through the city where citizens leave their refuse. Each collection point is made up of one or more refuse bins (with dimensions ranging from 1 to 3.2 m 3 ). Different trucks collect each type of waste and transport the waste to their final destination (a refuse dump, an incinerator or a recycling plant). One key factor for proper operation of collection systems is the design of appropriate collection routes for each type of garbage (fraction), which may or may not be separately collected (in some cases due to technical design constraints and different treatment processes). In Sant Boi de Llobregat each fraction is collected separately with a fixed periodicity (established by the municipality). Therefore, in this case the collection of each type of garbage can be considered as an independent but equivalent problem. In addition, the routes should be coherent with traffic rules, taking into account forbidden turns and traffic signals while minimizing collection costs.. Note that the considered problem is an arc routing one since in the actual circumstances of the problem roads are traversed completely, even if collection points are located in specific points of the roads. In addition the capacity of the trucks is limited. Thus, the basic nature of the considered problem is that of a capacitated arc routing problem (CARP). Even if they have not been so extensively studied as their node routing counterpart, there is a rather vast literature on arc routing problems (see [10] for a comprehensive text) and on the CARP in particular (see, for instance, [13, 17, 18, 2] ). Nevertheless, the problem that we address presents several characteristics that make it different from classical arc routing problems. These characteristics are mainly derived from traffic regulations. On the one hand, the underlying graph of the problem is a mixed one with both arcs and edges, since in reality some streets can be traversed in both directions whereas others have only one direction. To the best of our knowledge the CARP has not been studied on mixed graphs. On the other hand, at the junctions of the streets some turns will be forbidden, whereas other will be allowed. Again, to the best of our knowledge this is an issue that has not been considered for capacitated arc routing problems. For problems without capacities forbidden turns have been considered for routing and scheduling street sweepers in [3, 4] and for the Rural Postman Problem on mixed graphs with turn penalties in Corberán et al. [6] , where the authors propose a solution approach based on penalizing forbidden turns and post-processing illegal solutions. In [22] the formulation of the Capacitated Chinese Postman Problem is adapted to the context of the Canada Post Corporation and a general heuristic is presented to solve the problem under several hypothesis where, again, penalties are assigned to forbidden turns.. In this work we build a model for the Mixed Capacitated Arc Routing Problem (MCARP) where forbidden turns are taken into account in the set of constraints instead of penalizing forbidden turns via the objective function. We believe that this is more appropriate when the number of forbidden turns is high, like in the real problem that we have addressed. In Section 2 we describe the problem and in Section 3 we present the proposed model, which results from a transformation of the problem into a node routing one. In [16] Laporte proved that this strategy can be successful for addressing several classes of arc routing problems. It has also been applied to the Rural Postman Problem on mixed graphs with turn penalties in Corberán et al. [6] . Our solution approaches are presented in Section 4: Subsections 4.1 and 4.2 describe two constructive heuristics to obtain feasible solutions, and the neighborhoods that are explored in the Local Search, respectively. These elements are integrated within two heuristics based on ant colonies (see Dorigo and Stültze [9] ) that are presented in Subsection 4.3. Section 5 presents the results obtained when applying the algorithms of Section 4 to the city of Sant Boi de Llobregat. The routes generated with our heuristics are compared with the ones that were previously used, obtaining significant improvements. Finally, in Section 6, we present the conclusions of the work and we give some comments about the integration of the algorithms within SIRUS. In this paper we have presented the methodology that we have applied for the solution of an urban waste collection problem in Sant Boi de Llobregat within the metropolitan area of Barcelona (Spain). The nature of the problem is that of an arc routing one, but by means of a transformation it has been modeled in terms of an arc routing problem with additional constraints for forbidding some turns. Solutions to the problem have been obtained with heuristics based on ant colonies and different neighborhood strategies have been compared.. As reported in the Computational Experiments Section The obtained solutions improve considerably the ones that have been used so far both in terms of the operating cost and of acoustic contamination.. The applicability of the proposed procedures to real-life circumstances is subject to their integration within support systems to aid planners in decision-making. It should be possible to modify routes constantly to take into account changes on urban circulation. SIRUS [1] , was the software spplication designed in this case. The system provides a graphical tool to simplify different tasks required by the managers as the location of collection areas, allocation of refuse bins, routing, etc.

O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Runtime verification of authorization hook placement for the linux security modules framework We present runtime tools to assist the Linux community in verifying the correctness of the Linux Security Modules (LSM) framework. The LSM framework consists of a set of authorization hooks inserted into the Linux kernel to enable additional authorizations to be performed (e.g., for mandatory access control). When compared to system call interposition, authorization within the kernel has both security and performance advantages, but it is more difficult to verify that placement of the LSM hooks ensures that all the kernel's security-sensitive operations are authorized. We have examined both static and runtime analysis techniques for this verification, and have found them to be complementary. Static analysis is more complex to implement and tends to generate more false positives, but coverage of all type-safe execution paths is possible. Runtime analysis lacks the code and input coverage of static analysis, but tends to be simpler to gather useful information. The major simplifying factor in our runtime verification approach is that we can leverage the fact that most of the LSM hooks are properly placed to identify misplaced hooks. Our runtime verification tools collect the current LSM authorizations and find inconsistencies in these authorizations. We describe our approach for performing runtime verification, the design of the tools that implement this approach, and the anomalous situations found in an LSM-patched Linux 2.4.16 kernel. The Linux Security Modules (LSM) project aims to provide a Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. generic framework from which a wide variety of authorization mechanisms and policies can be enforced. Such a framework would enable developers to implement authorization modules of their choosing for the Linux kernel. System administrators can then select the module that best enforces their system's security policy. For example, modules that implement mandatory access control (MAC) policies to enable containment of compromised system services are under development.. The LSM framework is a set of authorization hooks inserted into the Linux kernel. These hooks define the types of authorizations that a module can enforce and their locations. Placing the hooks in the kernel itself rather than at the system call boundary has security and performance advantages. First, placing hooks where the operations are implemented ensures that the authorized objects are the only ones used. For example, system call interposition is susceptible to time-of-check-to-time-of-use (TOCTTOU) attacks [2] , where another object is swapped for the authorized object after authorization, because the kernel does not necessarily use the object authorized by interposition. Second, since the authorizations are at the point of the operation, there is no need to redundantly transform system call arguments to authorize kernel objects.. While placing the authorization hooks in the kernel can improve security, it is more difficult to determine whether the hooks mediate and authorize all controlled operations. The system call interface is a nice mediation point because all the kernel's controlled operations (i.e., operations that access security-sensitive data) must eventually go through this interface. Inside the kernel, there is no obvious analogue for the system call interface. Any kernel function can contain accesses to one or more security-sensitive data structures. Thus, any mediation interface is at a lower-level of abstraction (e.g., inode member access). In addition to mediation, it is also necessary to ensure that the proper access control policy (e.g., write data) is enforced for each security-sensitive operation. If there is a mismatch between the policy enforced and the controlled operations that are executed under that policy, unauthorized operations can be executed. We believe that manual verification of the correct authorization of a low-level mediation interface is impractical.. Much recent effort has focused on how static analysis tools may aid in the verification of various security properties [5, 12, 15] . As is the trend now, we expect that static analysis will be used where possible, and runtime analysis will be used to complete the analysis 1 . Thus, we are proceeding with the development of both static and runtime verification tools, and have found that static and runtime analysis have complementary features. Our static analysis approach enables comprehensive verification that the variables used ½ Consider the static checking of type-safe C code where possible and the runtime checking of other code used by Ccured [14] .. in security-sensitive operations have been authorized [18] . However, it is difficult to statically determine the authorization requirements that should be checked. This is because different data and control flows within functions may require different authorizations. One insight that we found useful is that many LSM authorization hooks are correctly placed, so inconsistencies in authorization requirements is often indicative of a problem. We built tools to enable users to find such inconsistencies and examine a wide variety of relationships in the runtime data to find problems.. In this paper, we present a runtime verification approach and tools to assist the LSM community and Linux kernel developers in verifying that the LSM authorization hooks completely authorize accesses. The runtime analysis approach involves: (1) instrumenting the Linux kernel to collect security relevant runtime events (e.g., major kernel events, such as system calls, LSM authorizations, and controlled operations) and (2) analyzing the collected data to identify potential errors. First, we extend GCC to perform analyses of its abstract syntax tree to add the necessary instrumentation to the Linux kernel. We have built kernel modules to collect the runtime events generated by the instrumentation. Second, we have designed a filtering language that enables the extraction of runtime events of interest for analysis. We have used this language to locate inconsistencies in authorizations for similar cases. Our analysis tools generate two different representations that we used to find inconsistencies: (1) authorization graphs that display the consistency between the execution of a controlled operations and its authorizations, and (2) sensitivity class lists that show the attributes of controlled operations to which the authorization consistency is sensitive. Using this approach, we have found three bugs in LSM hook placement in the file system that have since been fixed, and another anomaly that resulted in significant discussion. While the approach we use at present is not complete (i.e., some bugs may be missed), we are encouraged by our ability to find bugs using these tools. We demonstrate the use of these tools on a LSM-patched Linux kernel version 2.4.16.. The remainder of the paper is structured as follows. In Section 2, we define the general hook placement problem. In Section 3, we develop an approach to solving the general hook placement problem. In Section 4, we outline the implementation of the tools and discuss the analyses performed and their results. In Section 5, we conclude and describe future work. In this paper, we presented tools for assisting the Linux community in verifying the correctness of the Linux Security Modules (LSM) framework. The LSM framework consists of a set of authorization hooks placed inside the kernel, so it is more difficult to identify the complete mediation points. We leveraged the fact that most of the LSM hooks are properly placed to identify misplaced hooks. We used structure member operations on major kernel data structures as the mediation interface and collected the authorizations on these operations. By analyzing the output of a runtime logging tool, we identified the operations whose authorizations were inconsistent. We have analyzed the file system and some task operations and found some anomalies that could have been exploited. Working with the LSM community, these problems have since been fixed. For example, we found that some variants of fcntl enabled operations to be performed that were authorized in other cases. Ultimately, we found that runtime analysis is useful for verifying systems where a inconsistencies from the norm are likely to be errors. Further development of benchmarks for runtime analysis remains an challenge. We applied the December 10, 2001 LSM patch to the Linux 2.4.16 source and compiled the kernel using our modified version of GCC-3.0 3 . To create an execution log to analyze, we executed in parallel three instances of LMBench, the SAINT vulnerability tool (www.wwdsi.com/saint/), a kernel compile, some regular usage, and some test programs that we wrote as we became suspicious of anomalies. Since the effectiveness of runtime analysis depends on running enough code, the development of benchmarks that cover the enough of the interesting paths must be developed. For example, LMBench only runs about 20% of the kernel code. Also, our static analysis tool finds some other potential errors for which benchmarks should be written to determine if they can be exploited.. We have instrumented the kernel to collect controlled operations on the major kernel data structures: files, inodes, superblocks, tasks, sockets, and skbuffs. Thusfar, we have only done a detailed analysis on the file system authorizations, and an initial analysis on task authorizations. Since the file system is fairly well-understood, we did not expect a large number of anomalies, but we found some nonetheless.. Member Sensitive (multiple system calls) : We found that there is no authorization hook in the function setgroups16, but that we can reset the task's group set. An authorization protects this operation in setgroups. This hook was missed because these backwards ABI-compatible 16-bit task operations, such as setuid16 and setchown16 usually convert their 16-bit values to 32-bit values and call the current versions that do contain authorizations. However, since setgroups16 sets an array, it is easier not to convert the array, so the current version (that contains a hook) is not called. Note that there is no setgroups16 call in the current version of libc, so we had to write an assembler program to perform this exploit.. Member Sensitive (single system call): The f_owner.pid member of struct file tells the kernel which process to send signals to regarding IO on this file. Setting this field is authorized by file_ops->set_fowner if the user tries to set it directly via fcntl(fd, F_SETOWN, pid_-owner). However, if a user removes a lease from a file via fcntl(fd, F_SETLEASE, F_UNLCK), the owner is set to zero without the authorization being performed. Furthermore, a process can set the owner of a Universal TUN device (drivers/net/tun.c) to itself without the authorization being performed. To achieve this, the process calls ioctl(fd, F_SETFL, FASYNC) on an open, attached, TUN device.. Member Sensitive (single system call): During our investigation of the sensitivity of filp.f_owner described above, we we found that access to filp.f_owner.signum (the signal that should be sent upon IO completion) can be set without the authorization via fcntl(fd, F_SETSIG, sig). Recently, static analysis to verify security properties has shown promise. First, existing program analysis tools have been used to find common security errors, such as buffer overflows and printf vulnerabilities [12, 15, 17] . Second, specialized tools have been developed for finding security vulnerabilities, such as xgcc [5] , ITS4/RATS [16] , and MOPS [3] .. Program analysis tools are based on formal properties of programming languages, so they are can be used for complete analysis (i.e., no false negatives). However, program verification is computationally expensive, so simplifications in these models can result in conservative analyses (i.e., many false positives). Also, program analysis tools require a significant amount of effort to annotate the source, so that useful analysis models can be generated.. Specialized analysis tools focus on specific types of bugs with simplified source annotation. Engler et al enables extension of GCC, called xgcc, to do source analyses, which they refer to as meta-compilation [5] . A rule language, called metal, is used to express the necessary analysis annotations in a higher-level language. Since the rules match multiple statements, the amount of annotation effort is reduced. A variety of software bugs, including security vulnerabilities, have been found by this tool [1] .. Most of the specialized analysis tools lack completeness (i.e., may result in false negatives), but MOPS specifically aims for ease of specification and completeness of analysis [3] . Using MOPS, security properties are expressed as finite state automata and programs are represented as pushdown automata. Given security property analyses that can be represented in this model, a complete analysis is possible under certain assumptions.. In another effort, we use one program analysis tool, CQUAL [6] , in an approach to finding LSM hook placement bugs statically [18] . Using GCC analysis to automate CQUAL annotation, we can then perform a CQUAL analysis that verifies that all controlled operations are mediated by at least one LSM hook. In general, we also want to verify that a controlled operation is only run when its required authorization hooks have been checked. While it is possible to verify this in CQUAL, we need to find the required authorizations. None of the existing static analysis tools help find the required authorizations, so we have built the runtime collection and analysis tools described in this paper. A Java static analysis tool, called Jaba [11] , has been used to collect the actual authorization on controlled operations, but it has two shortcomings: (1) it does not analyze the C code of the Linux kernel and (2) it does not provide guidance about whether the authorizations made were correct. Our analysis tools collect the authorizations made and enable examination of the authorizations of particular controlled operations for inconsistencies. We found such an approach useful for finding missing LSM hooks, because we could identify many cases where the same authorizations are expected.. Another related problem is the certification of systems. Historically, the Orange Book [13] was used for guidance in the construc-tion of secure operating systems, but this is now being supplanted by the Common Criteria [8] . However, the certification task is ad hoc and laborious, and has generally not been successful in improving the security of commonly-used operating systems. Gutmann argues in his thesis [7] that certification approaches, including formal verification tools, are doomed to failure unless they represent concepts at the level of the source code. Gutmann also advocates a combination of static and runtime analyses. The approach that we use differs from certification in the sense that it checks for particular errors rather than providing a top-down assurance that the overall system meets its requirements. An interesting research question is whether a sufficient breadth and depth of such checks could provide a confidence comparable to certification. Unlike certification, such confidence could be maintained as the source code evolves. Complete authorization is verified by analyzing (offline) a kernel execution log. This section describes the implementation of the tool that creates this log, the implementation of the log filtering tool used to prepare and display analysis data, and the results of our analysis thusfar. Table 2 shows the information collected during runtime analysis. Controlled operations are identified by the tuple (instruction pointer, object type, member, access). A controlled operation ID is assigned to each unique combination. Authorizations are uniquely identified by (LSM hook, policy operation). Like controlled operations, a unique authorization ID is assigned to each. Function entry and exit are recorded as well. The function entry address uniquely identifies the function.

B B O O O B O O O O O B O O B B O O O O O O O O B O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O B I O O B O B B O B I O O O O O O O O O O O O B O O B B O O O O O O O O O O O O O O O O O O O O O O B B B B O O O O O O O O O O O O O O O O B B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I B O O O O B O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O B B B O O O O O O B I O O O O B I O O O B O O O O O O O O O O O O B O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O O O O O O O O O O B O O O B O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B B O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Representing concerns in source code A software modification task often addresses several concerns. A concern is anything a stakeholder may want to consider as a conceptual unit, including features, nonfunctional requirements, and design idioms. In many cases, the source code implementing a concern is not encapsulated in a single programming language module, and is instead scattered and tangled throughout a system. Inadequate separation of concerns increases the difficulty of evolving software in a correct and costeffective manner. To make it easier to modify concerns that are not well modularized, we propose an approach in which the implementation of concerns is documented in artifacts, called concern graphs. Concern graphs are abstract models that describe which parts of the source code are relevant to different concerns. We present a formal model for concern graphs and the tool support we developed to enable software developers to create and use concern graphs during software evolution tasks. We report on five empirical studies, providing evidence that concern graphs support views and operations that facilitate the task of modifying the code implementing scattered concerns, are costeffective to create and use, and robust enough to be used with different versions of a software system. Useful software systems need to change [Belady and Lehman 1976] . In general, the effort spent repairing, adapting, and enhancing software systems to ensure their continued value is significant because for each change task, a developer has to understand the existing software, modify it, and then validate the changes [Boehm 1976 ].. A software modification task often addresses several concerns. The term concern is loosely defined to represent anything that stakeholders of a software project may want to consider as a conceptual unit. Typical concerns in a software project include features, nonfunctional requirements, design idioms, and implementation mechanisms (e.g., caching). To make it easier to evolve the code base, software engineers generally try to keep concerns separate as they build a system [Parnas 1972 ]. Unfortunately, in practice, it is not possible to separate all concerns in a system due to a variety of reasons, including inadequate initial design, the limitations that programming languages impose on the decomposition of software systems [Kiczales et al. 1997; Tarr et al. 1999] , the emergence of unforeseen concerns as a system evolves, and the decay of code structures following repeated changes [Eick et al. 2001; Belady and Lehman 1976; van Gurb and Bosch 2001] . As a result, many concerns end up scattered and tangled throughout the code of a system [Tarr et al. 1999] . Scattered concerns make it difficult for developers to reason about which pieces of code interact to implement a concern, and about how different concerns interact with each other. An incomplete understanding of a concern prior to a program change can cause a developer to make incorrect or inefficient modifications [Letovsky and Soloway 1986; Robillard et al. 2004] , or modifications that do not respect an existing design [Parnas 1994 ].. To make it easier to modify concerns that are not well modularized, we propose an approach in which the implementation of concerns is documented in artifacts, called concern graphs [Robillard and Murphy 2002] . Concern graphs are abstract models that describe which parts of the source code are relevant to different concerns. A concern graph abstracts the implementation details of a concern by recording the key structure implementing it. By recording structure, a concern graph explicitly documents the relationships between the different sections of code that play a role in the implementation of a concern. A concern graph is based on a program model that can be extracted automatically from the source code or similar forms. By staying close to the source in its abstraction, a concern graph can remain tightly linked with the code.. To support our approach, we have created a tool, called FEAT, that helps developers build concern graphs semiautomatically as they investigate a program. Once a concern graph is available, the tool allows a developer to view only those parts of a software system that are relevant to a concern, to view them easily in the context of the corresponding source code, and to perform analyses on the concern description (e.g., to see how two concerns overlap in the code).. We claim that concern graphs and their supporting technology are:. -functional, supporting views and operations that facilitate the task of modifying the code implementing scattered concerns; -cost-effective, as they can be created as part of normal program investigation activities; and -robust, as they can tolerate changes to a code base, and can thus be reused with different versions of a system. This property further reduces the relative cost of the approach, since a single concern graph can be used to support multiple tasks.. The need to document scattered concerns so as to support software evolution was identified by Soloway et al. [Letovsky and Soloway 1985; Soloway et al. 1988] . Unfortunately, traditional documentation such as that proposed by Soloway suffers from two principal drawbacks: It is costly to produce and difficult to maintain consistently with the source code. Program understanding and reverse engineering [Chikofsky and Cross II 1990] approaches have also been developed to help a developer discover the code related to a program change task (e.g., cross-reference search engines [Chen et al. 1990; Goldberg 1984; Object Technology International 2001; O'Brien et al. 1987; Sanella 1983] , or feature location techniques [Wilde et al. 1992; Wilde and Scully 1995] ). These approaches can help a developer track down the code relevant to a concern, but have a limited capacity to help developers preserve, view, and manage the knowledge discovered. As such, concern graphs not only help developers find and view information about the implementation of concerns in source code, but also enable them to preserve the information, opening new possibilities for linking documentation to source code. Aspect-oriented programming (AOP) approaches [Kiczales et al. 1997] aim to modularize crosscutting structure explicitly in programs. Concern graphs complement AOP by providing an inexpensive way to group scattered code, without requiring changes to the code base.. We have implemented our approach for the Java programming language, and performed multiple empirical studies to validate whether our approach meets the aforementioned claims. In Section 2 we motivate this research by presenting a small practical example. In Section 3 we present the formal model for concern graphs which is the foundation for our approach. Section 4 is a description of the existing tool support. In Section 5, we present the validation of the approach. Section 6 briefly discusses open questions arising from the research. We conclude with a discussion of related work in Section 7 and a summary of the article in Section 8. Evolving a software system typically requires a developer to locate and understand concerns that are not adequately separated in the code. To help developers cope with software change tasks involving scattered concerns, we propose to explicitly model the implementation of concerns in software engineering artifacts. We developed a formal representation for concerns in source code, called concern graphs. Concern graphs are an abstraction of the implementation of a concern that can be mapped onto the actual source code of a system.. To experiment with concern graphs, we have developed a tool called FEAT, which allows developers to iteratively build concern descriptions as the source is investigated, to view the code related to a concern, and to perform analyses on the concern representation. Concern graphs also support a specialized mechanism, implemented in FEAT, that enables the detection, management, and repair of inconsistencies between a concern graph and an actual code base.. Using FEAT, we have evaluated the costs and benefits of concern graphs in a series of case studies involving the evolution of five different systems of differing size and style. The results show that concern graphs are inexpensive to create during program investigation, support views and operations that facilitate the task of modifying the code implementing scattered concerns, and are robust enough to be used with different versions of a system.. In conclusion, the concern graph approach is an integrated and practical solution to the problems of finding, viewing, and preserving information about the implementation of concerns in source code. Interesting issues and questions arose during the development and validation of the concern graphs approach that open the door to additional investigation. To illustrate the challenge of finding and understanding the implementation of scattered concerns and to introduce our approach, we describe what a developer must consider and remember in order to extend a feature in the JHotDraw drawing program.. 1 JHotDraw is implemented in Java and comprises about 16kLOC of source code (not including comments and blank lines). In release 5.3 of JHotDraw, two different binary file formats are supported to store drawings to disk. The task of our developer is to modify the "save" feature of JHotDraw to support an additional file format. In this case, our concern of interest, SAVE FEATURE, corresponds to the code implementing the saving of drawings to disk.. Conceptually, the SAVE FEATURE concern is trivial. However, its implementation is not: The code implementing SAVE FEATURE is scattered throughout at least 35 classes, and interacts as well as intersects with other concerns, as the following three examples demonstrate.. -To understand where the action to save a drawing is triggered, the developer must discover the implicit invocation mechanism responsible for creating and triggering actions based on menu selections. In JHotDraw this mechanism, which we call the COMMAND concern, is implemented as a variation of the Command design pattern [Gamma et al. 1995] , and involves at least 15 methods scattered in six classes (not counting concrete commands that have nothing to do with SAVE FEATURE). -To understand how different storage formats are managed, the developer must investigate the details of the storage management mechanism. This STORAGE MANAGEMENT concern involves many different interactions between at least 15 methods scattered in four classes (e.g., StorageFormatManager and StorageFormat). These interactions are necessary to carry out such tasks as registering different file formats with a file chooser widget, and obtaining the writer object representing the correct storage format. -To understand how to actually write a drawing to a file, a developer will need to realize that there exists a Storable interface, and that all the objects in a drawing that can be written to a file must implement a write method, which writes to a special type of object called StorableOutput. In version 5.3 of JHotdraw, the interface Storable is implemented by 24 different classes. We refer to this mechanism as the WRITING concern. In total, it is implemented by about 39 methods scattered in 31 different classes.. The challenge of modifying SAVE FEATURE stems from having to discover, understand, and keep track of separate, but interacting, subsets of the code associated with different high-level concepts (i.e., concerns). In practice, the STORAGE MANAGEMENT, COMMAND, and WRITING concerns all interact in the source code. For the change task, the developer will likely need to understand the details of each concern, including their interactions. As an example of interaction between the three concerns, Figure 1 shows the code of a method called promptSaveAs(). This method is the callback executed by the save command described in COMMAND, implements USER INTERFACE code (a fourth concern, lines 4 and 7), uses the STORAGE MANAGEMENT mechanism (lines 5, and 9-15), and calls into code implementing the WRITING concern (lines 14-15). To allow developers to capture subsets of the system relevant to concerns of interest, and to be able to work with the system in terms of the concerns, we developed a representation called concern graphs. A concern graph is built interactively while a developer investigates a program, shows only that subset of the program associated with the implementation of a concern, and allows developers to perform a number of analyses (e.g., to see how different concerns interact in the source code).. Figure 2 presents a partial view of a concern graph for SAVE FEATURE as displayed in FEAT, a tool we developed to support the concern graph approach. A concern graph represents the subset of a software system associated with a specific concern. Concern representations can be organized in a hierarchy (left window in Figure 2 ), where each subconcern specializes the parent concern. For example, the COMMAND concern represents that subset of the program Selecting any concern or subconcern on the left displays the subset of the program associated with the concern. For example, in Figure 2 , the structure corresponding to STORAGE MANAGEMENT is displayed in the right window; only seven methods are visible from the 91 members and nine inner classes that make up the DrawApplication class. From this simplified view, it is also possible to perform other operations on concern graphs, such as to drill down to the source code and perform cross-reference queries and interaction analyses. For example, if the developer does not remember the role that method open plays in the implementation of the concern, selecting this method in FEAT will display (in a separate view) a link labeled "accessing DrawApplication.fStorageFormatManager." This link provides explicit information about the connection of open to the concern. Clicking on the link will automatically display and highlight the access to fStorageManager, which is buried in one of the 45 lines of the open method. Other aspects of FEAT that support software modifications are detailed in Section 4. Early empirical evidence that scattered concerns pose problems to programmers was collected by Soloway et al. during different studies of professional programmers [Letovsky and Soloway 1986; Soloway et al. 1988] . In one study conducted at NASA's Jet Propulsion Laboratory, Soloway et al. [1988] observed that the programmers who did not implement a correct modification to a small system "failed to understand the casual interactions inherent in one of the key delocalized plans" [Soloway et al. 1988 [Soloway et al. , p. 1262 . To address the difficulty of performing maintenance on code involving delocalized plans (in other words, scattered concerns), the researchers propose that programmers produce explicit documentation detailing delocalized plans in programs. Their initial approach is a form of paper documentation, where source code is presented in parallel with pointers linking the code to other relevant sections of a program, and detailing the rationale for different design and implementation decisions. Although the idea of Soloway et al. [1988] is based on sound empirical observations, free-form documentation is effort-intensive to produce and maintain. Since this early work on delocalized plans, a number of approaches have been proposed to document the implementation of scattered concerns in software systems. In the rest of this section, we describe the body of work in software engineering whose explicit focus is the representation of concerns in source code.. Descriptions of scattered concerns can be captured as virtual files. In software development environments, the idea of a virtual file is to present various segments of source code and other system documentation relevant to a task as a single unit. For example, in the Desert environment [Reiss 1996 ], a developer can load, edit, and save a virtual file consisting of fragments of other source files. Stellation [Chu-Caroll et al. 2002; Chu-Carroll and Spenkle 2000 ] is a fine-grained software configuration management system that supports method-level storage management. Using a concept similar to Desert, Stellation supports virtual source files using a typed aggregation mechanism that collects different program elements and other artifacts (such as test cases) in a single unit for the purpose of configuration management. The proposal for Stellation includes the possibility of specifying aggregates either explicitly or as the result of a query. Virtual files could be used to document scattered code that implements a concern. However, to compose a virtual file, a developer must already know about the location of the code implementing a concern. The mechanisms proposed for Desert and Stellation also do not include support for tolerating and managing inconsistencies between a virtual file and the source code.. A number of approaches support the representation of concerns based on characteristics of the program text forming a software system. Conceptual modules [Baniassad and Murphy 1998 ] allow developers to specify concerns as a collection of scattered lines of source code, and to perform analyses on the resulting modules. Because conceptual modules are defined in terms of lines of source code, they cannot be reused with different versions of a system. The Aspect Browser is a tool proposed to help developers find concerns using lexical searches of the program text [Griswold et al. 2001] . Concerns found in this fashion can be stored and viewed at different times to support program evolution tasks. Aspect Browser uses the Seesoft [Eick et al. 1992] concept and a map metaphor to graphically represent the location of code implementing concerns in the context of the entire code base. The aspect mining tool (AMT) [Hannemann and Kiczales 2001] is conceptually similar to Aspect Browser, but supports additional queries based on types. The Aspect Browser and AMT can be used both for finding and documenting concerns. However, because they only support the specification of concerns based on lexical matches to regular expressions and the use of types, their expressive power is limited. The textoriented approach also limits the tools' ability to capture relationships between scattered program elements explicitly.. Other approaches support the representation of concerns in source code based on structural properties in ways that provide cost-benefit trade-offs that are different from concern graphs. Intentional Views [Mens et al. 2002; allow developers to specify different views of a system that reflect some form of commonality, which can include relevance to the implementation of a concern. Intentional Views are specified explicitly using a declarative programming language and thus offer more flexibility, but at a higher cost to developers, who need to explicitly form the queries describing the views. Intentional Views are also generative and do not store the subset of a program generated from a view, so they do not allow the type of reasoning about changes to the source code that concern graphs support. Cosmos [Sutton and Rouvellou 2002] is an approach that has been proposed to model concerns in terms of high-level system characteristics. As opposed to concern graphs, the Cosmos model currently does not support maintaining links between concern descriptions and the corresponding source code. The concern manipulation environment (CME) [Harrison et al. 2004 ] is a project whose goal is to integrate support for creating and evolving aspect-oriented software across the life cycle of a system. CME includes a concern explorer tool that can be used to describe concerns using queries in a way similar to FEAT. FEAT and CME followed parallel research courses and share many similarities. However, the research focus for each project is different. While CME is aimed at providing a unified way to represent concerns across different types of software engineering artifacts, FEAT focuses on linking concern descriptions and other artifacts with the source code. As a result, CME is more open (supporting, e.g., relations between elements not found in the code), while FEAT supports a higher level of reasoning about the links between concerns and source code (e.g., concern interactions expressed as concerns, and analysis of inconsistencies between a concern description and a version of the source code). Besides the user interface described in the previous sections, the architecture of FEAT is comprised of two components: the model and the analyzer.. The model component supports the runtime representation of a concern graph, including loading and storing a concern graph, and tolerating inconsistencies between a concern graph and the source code through a mechanism of pollution markers inspired by the work of Balzer [1991] . It is important to note that the model component only represents a concern graph model, and does not contain any of the program information forming the program model. The latter is stored in the analyzer component.. The responsibility of the analyzer component is to produce a model for a program to support queries on this model, and to support mapping fragments to the corresponding source code. In our current version of FEAT, a user can only associate a single concern graph to a given program model. As a consequence, a user wishing to work with multiple concerns must do so in the context of a single concern graph. In practice, this simplifying design decision seems to have little impact on the usability of FEAT and we have not yet received any request to change this aspect of the tool.. The analyzer component is designed to optimize the speed of FEAT queries at the cost of an initial model extraction time. The FEAT analyzer produces a model of a program by executing a single pass through the abstract syntax tree (AST) of every Java file in the project associated with a concern graph. When scanning the AST of Java source files, the analyzer searches for instances of the relations supported by the mapping function. When a relation between two elements is identified, the analyzer stores both the relation and its transpose in an in-memory database. Our choice of a source code-based approach for the production of a program database stems from the convenience offered by Eclipse's incremental compiler, and is independent from any fundamental properties of concern graphs. For example, our initial FEAT prototype used bytecode analysis to produce the program database [Robillard and Murphy 2002] . It is important to note that the use of source code has a limited practical impact on the overall approach, since it is still possible to add any program element to a concern graph, whether it is defined in source code or in bytecode. Our program database also includes relations from source elements to bytecode elements. In the case of relations from bytecode elements (which are not supported), we presume that these are of limited applicability, since users of FEAT cannot change the corresponding code. There are a few cases (such as callbacks from framework methods) where the information could be useful, but so far, we have not found this to be a severe enough limitation to warrant redesign of the analyzer component.. • M. P. Robillard and G. C. Murphy The static analysis required to extract the model of a program, and the size of the database produced, both impose practical limits on the size of the programs analysable by FEAT. To allow FEAT to work on large programs, developers can control the scope of the analysis. When creating a new concern graph for a project, a developer can select from a list of all source packages for a project just those that should be included in the program model database. Elements declared in packages left out of the analysis can still be viewed and used in FEAT, but their source code is not analyzed. This flexibility allows users to remove basic libraries and other elements that they know are not involved in the concerns they are analyzing, reducing the storage and computation load on the tool.. Because the analyzer performs a single pass through the source code, the time required for model extraction increases linearly with the size of the source code analyzed, with an origin close to zero and a slope of just under 1.1s/kLOC.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An Experimental Analysis of the Dependence among Codeword Bit Errors in ECOC Learning Machines One of the main factors affecting the effectiveness of Error Correcting Output Coding (ECOC) methods for classification is the dependence among the errors of the computed codeword bits. We present an extensive experimental work for evaluating the dependence among output errors of the decomposition unit in ECOC learning machines. In particular, we apply measures based on mutual information to compare the dependence of ECOC Multi-Layer Perceptron (ECOC MLP), made up by a single multi-input multi-output MLP, and ECOC ensembles made up by a set of independent and parallel dichotomizers (ECOC PND). Moreover, the experimentation analyzes the relationship between the architecture, the dependence among output errors and the performances of ECOC learning machines. The results show that the dependence among computed codeword bits is significantly smaller for ECOC PND, pointing out that ensembles of independent parallel dichotomizers are better suited for implementing ECOC classification methods. The experimental results suggest new architectures of ECOC learning machines to improve the independence among output errors and the diversity between base learners. Error Correcting Output Coding (ECOC) [19] is a an Output Coding (OC) decomposition method [30, 26] that has been successfully applied to several classification problems [1, 11, 20, 6, 33] . OC methods decompose a multiclassclassification problem in a set of two-class subproblems, and then recompose the original problem combining them to achieve the class label.. ECOC methods present several open problems such as the tradeoff between error recovering capabilities and learnability of the dichotomies induced by the decomposition scheme [2, 43] . A connected problem is the analysis of the relationship between codeword length and performances [20] , while the selection of optimal dichotomic learning machines and the design of optimal codes for a given multiclass problem are open questions that are still subject to active research [16] .. Another issue tackled by several works [24, 21] is the relationship between performances of ECOC and dependence among output errors. In the framework of coding theory, Peterson [34] has shown that the error recovering capabilities of ECOC codes hold if there is a low correlation among codeword bits. We qualitatively identified the dependence among output errors as one of the factors which influences the effectiveness of ECOC decomposition methods [27] . In that work we hypothesized a higher dependence among codeword bit errors in monolithic Error Correcting Output Coding (ECOC monolithic) [19, 27] compared with ECOC Parallel Non linear Dichotomizer (ECOC PND) [28] learning machines, considering that ECOC monolithic share the same hidden layer of a single MLP, while PND dichotomizers, implemented by a separate MLP for each codeword bit, have their own layer of hidden units, specialized for a specific dichotomic task.. The main goal of the paper consists in understanding the relationships between the performances and the dependence between codeword bit errors in ECOC learning machines. In particular our aim is to analyze and to unravel the relationships between the architecture of the ECOC learning machines, the dependence among the errors of the decomposition unit and the resulting performances. This analysis requires a quantitative evaluation of the dependence among codeword bit errors. To this purpose we apply mutual informationbased measures of dependence among output errors proposed in [29] , interpreting the dependence among the output errors as the common information shared among them. These measures assess the dependence among the output errors considering their probability distributions, and in this sense they are more refined measures of dependence compared with the standard index of correlation or the rank order correlation coefficient. In particular, they can offer insights into the dependence and the probability distribution of the er-rors and can also be used to compare the dependence among output errors between different learning machines in order to select a model well-suited to a particular learning problem.. The proposed analysis of the dependence among codeword bit errors represents a novel application of mutual information to a machine learning problem. Information theory and in particular mutual information had been applied to several machine learning problems, such as modeling of self organized systems and feature maps [25, 9] , feature transformation and selection [7, 40] , image processing [8, 41] , independent component analysis [14] , evaluation of the relations between output independence and complementariness in multiple classifier decision systems [36] .. In this paper an extensive experimental comparison between different architectures of ECOC learning machines is accomplished, evaluating the relationships between the accuracy of the predictions and the dependence between codeword bit errors. In particular, ECOC monolithic [19, 27] and ECOC PND [28] are compared using synthetic and UCI data sets [31] , and a specific test of hypothesis [29] is applied to evaluate whether a significant statistical difference in the dependence among the codeword bit errors between the two ECOC learning machines does exist.. The paper is organized as follows. In the next section we outline ECOC methods and their main related open problems. In Sect. 3 we summarize the main characteristics of the measures based on mutual information to evaluate the dependence among output errors. Sect. 4 presents the experimental setup, the results and the discussion. The conclusions summarize the main results and the incoming developments of this work. , was generated using NEURObjects, a C++ software library for neural networks development [44] , and the other three, glass, letter and optdigits were taken from the UCI machine learning repository of Irvine [31] . The synthetic data set d5 is made up by five three-dimensional classes, each composed by two disjoint clusters of data: the data points for each class were drawn from two normal distributions with equal probability and different diagonal covariance matrix. The main characteristics of the data sets are shown in Tab. 2 In order to perform training and testing of the considered learning machines, we applied multiple runs of different random initializations of the weights using a single pair of training and testing data sets and k-fold cross validation [13] methods. The results are summarized in Tab. 3: Errors on the test set are expressed as percent rates, and for each data set the minimum (min), average (mean), and standard deviation (stdev) of the error is given.. After the training, we used only the outputs of the decomposition units of the learning machines. Then we computed the errors, obtaining the matrices Using these error data we computed and compared the mutual information error I E (eq.5) and the mutual information specific error I SE (eq.6) among all the outputs of the learning machines. Then, we computed and compared the mutual information error matrices R, the mutual information specific error matrices S (Sect. 3), and the their relative global indices Φ R and Φ S (eq. 7 and 8).. We used NEURObjects [44] , both to train the learning machines and to evaluate the dependence among the output errors.. We compared the dependence among output errors of ECOC monolithic and ECOC PND learning machines varying the structure (number of hidden units), the number of intervals (bins) of the output errors, and the values of the output error tolerance δ (Sect. 3). For each data set and for a fixed number of hidden units we have considered all the combinations of δ ∈ {0.1, 0.2, 0.3, 0.4} with the number of intervals bins ∈ {2, 3, 4, 5, 6}, for a total of 20 pairs of (δ, bins).. For the data set d5 we used 11 different structures for the learning machine, varying the number of hidden units between 5 to 50, yielding to 11 × 20 = 220 evaluations of I E , I SE , Φ R and Φ S both for ECOC monolithic and ECOC PND learning machines. For the UCI data sets glass, letter and optdigits we used only 2 different structures, using, respectively, 5 and 9, 120 and 140, 60 and 70 hidden units, yielding to 2 × 20 = 40 evaluations of the mutual information error based quantities both for ECOC monolithic and ECOC PND learning machines.. The ECOC codes generated for letter and optdigits data sets lead up to learning machines with respectively 30 and 14 outputs. elements, requiring a large amount of data, not available for these data sets, in addition to an intractable amount of space and time computational resources. It is worth noting that this problem is a form of curse of dimensionality [10] , that can be avoided computing only the global pairwise indices Φ R and Φ S . To evaluate the dependence between specific pairs of output errors, we also compared R and S matrices between ECOC monolithic and PND considering a single triplet structure, numbers of bins and δ. The effectiveness of ECOC decomposition methods depends on many factors, including the similarity of the ECOC codewords, the accuracy of the dichotomizers, the complexity of the multiclass learning problem, the design of learning machines implementing the decision units, and the dependence among codeword bits.. While some of these problems have been tackled elsewhere [24, 11, 16, 28, 2, 43] , the proper design of ECOC learning machines and the quantitative evaluation of the dependence among codeword bits have not been adequately addressed.. In this paper we have presented an extensive experimental work to evaluate quantitatively the dependence among codeword bits errors in ECOC learning machines. In particular, we have proposed and used measures based on mutual information to compare the dependence among output errors between ECOC monolithic and ECOC PND learning machines.. The measurements of the mutual information error I E , the mutual information specific error I SE and the mutual information error matrix indices Φ R and Φ S show that ECOC PND present a lower dependence among the output errors of their decomposition unit compared with the output errors of the corresponding ECOC monolithic MLP.. We have also analyzed the relationship between performance and dependence among output errors, showing that the design of ECOC learning machines affects this relationship. In fact, the results show that monolithic architectures are affected by a higher dependence among codeword bit errors leading to a higher generalization error. Our experimental work suggests that a low dependence can be achieved implementing the decomposition unit through an ensemble of parallel and independent dichotomizers, such as the dichotomic MLP proposed in our experiments, or other suitable dichotomizers such as decision trees [35] or support vector machines [15] .. Future developments of this work should consist in studying quantitatively the dependence among output errors in ECOC learning machines architectures that can improve the diversity between the dichotomizers implementing the decision unit. In particular, we shall quantitatively study how boosting methods [37] can increase the diversity among the dichotomizers and the independence among output errors in ECOC learning machines. We used four different data sets: the first one, d5

O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fuzzy methods for case-based recommendation and decision support Abstract The paper proposes two case-based methods for recommending decisions to users on the basis of information stored in a database. In both approaches, fuzzy sets and related (approximate) reasoning techniques are used for modeling user preferences and decision principles in a flexible manner. The first approach, casebased decision making, can principally be seen as a case-based counterpart to classical decision principles well-known from statistical decision theory. The second approach, called case-based elicitation, combines aspects from flexible querying of databases and case-based prediction. Roughly, imagine a user who aims at choosing an optimal alternative among a given set of options. The preferences with respect to these alternatives are formalized in terms of flexible constraints, the expression of which refers to cases stored in a database. As both types of decision support might provide useful tools for recommender systems, we also place the methods in a broader context and discuss the role of fuzzy set theory in some related fields. So-called recommender systems (Kautz, 1998 ) (see Resnik & Varian (1997) for a concise introduction) have been a focus of research in recent years. Roughly speaking, such systems assist people without sufficient personal experience in making choices between alternatives. For example, such systems may use a database about user preferences in order to predict additional topics or products a (new) user might like (Breese, Heckerman, & Kadie, 1998) . So-called collaborative filtering (Goldberg, Nichols, Oki, & Terry, 1992) allows for personalization in e-commerce by exploiting similarities and dissimilarities among customers' preferences (Lin, Alvarez, & Ruiz, 2002) .. Recommender systems employ methods from various fields including, among others, database technology and case-based reasoning. In both fields, fuzzy set theory turned out as an appropriate theoretical foundation for several kinds of extensions and further developments. This is mainly due to some distinctive features of fuzzy sets, notably their ability to handle concepts such as similarity, preference and uncertainty in a unifying way, the possibility of representing graded degrees of truth or acceptance, and the capability of approximate and plausible reasoning. These are valuable features when retrieving objects stored in a database, extrapolating from experience given in the form of stored cases or, more generally, when making recommendations to users.. Our focus in this paper is on case-based decision support, which can serve as a basis for a special type of recommender system: Suppose that a user having to make a decision has access to cases stored in a database, and that the user's preferences for alternative decisions depend, in a way to be specified, on these cases. In other words, the user does not (at least not only) have, say, absolute preferences. Rather, the preferences are expressed in a relative way with respect to the stored cases. Given this type of preferences, the system should suggest one or several among the potential alternatives on the basis of the information provided by these cases.. Section 2 recalls some basic concepts of fuzzy sets. To put the contents of the current paper into context, Section 3 provides a brief overview of applications of fuzzy sets in case-based decision support and related fields. Two different types of case-based decision support are then discussed in detail in Sections 4 and 5, respectively. Section 4 extends work that has been presented in Dubois et al. (1997) . The approach proposed in Section 5 is a generalization of case-based prediction as introduced in Dubois, Hü llermeier, & Prade (2002) .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Existence of local strong solutions for motions of electrorheological fluids in three dimensions We prove for three-dimensional domains the existence of local strong solutions to systems of nonlinear partial differential equations with p(·)-structure, p ∞ ≤ p(·) ≤ p 0 , and Dirichlet boundary conditions for p ∞ > 9 5 without restriction on the upper bound p 0 . In particular this result is applicable to the motion of electrorheological fluids. Electrorheological fluids are special viscous liquids that change their viscosity rapidly when an electric field is applied. The motion of such fluids is governed by a system of nonlinear partial differential equations which read (cf. [1, 2] ). 1 div (E + P) = 0, curl E = 0,. ρ 0 ∂v ∂t. where E is the electric field, P the polarisation, ρ 0 the density, v the velocity, S the extra stress tensor, φ the pressure and f the mechanical force. The above system has to be supplemented with constitutive relations for P and S and with boundary and initial conditions. According to the model in [1] the extra stress S is given by S(D, E) = α 21 . Here α i j are material constants, p is a given material function with 1 < p ∞ ≤ p(|E| 2 ) ≤ p 0 and D is the symmetric velocity gradient, i.e. 2D(v) = ∇v + (∇v) t . These equations with p(·)-structure are one example of systems with non-standard growth conditions. In the context of calculus of variations, structures with variable exponents were first considered in [3, 4] . In recent years there has been some substantial progress in the existence and regularity theory of problems with p(·)-structure (cf. [5] [6] [7] [8] [9] [10] ), in the theory of Lebesgue and Sobolev spaces with variable exponents (cf. [11] [12] [13] [14] [15] ), the theory of singular integrals in these spaces (cf. [7] [8] [9] [10] ) and the development of models with p(·)-structure, e.g. in mathematical physics and image processing. If we choose in (1) a linear relation for P, i.e. P = χ E E, where χ E is the dielectric susceptibility, the system decouples in the mechanical and electric variables. Since Maxwell's equations are well understood (cf. [16, 2] ) we will only treat the system (1b). Moreover, we restrict ourselves to steady situations, Dirichlet boundary conditions and homogeneous electrorheological fluids. 2 Hence we study the existence of strong solutions to the system. It is shown in [2] that S has under suitable assumptions on the coefficients α i j and on the material function p ∈ W 1,∞ the following properties: strong monotonicity. , the existence of a strong solution up to the boundary was proved by Málek, Nečas and Růžička in [23] .. Theorem. Let Ω ⊂ R 3 be a bounded domain, let S fulfil Assumption 1, f ∈ L q (Ω ) with 3 q = min(2, p ∞ ) and p ∞ > 9/5 and E ∈ W 1,∞ (Ω ). Then there exists a local strong solution of the problem (3), i.e. v ∈ W 2,p(·) loc. Further, for all U ⊂⊂ Ω we have. In order to prove this theorem we construct approximate solutions v λ as solutions of problem (3) with S substituted by S λ . For this approximate problem it holds that:. Proof. We get the existence of a weak solution v λ ∈ V 2 with the help of Brezis' theorem on pseudomonotone operators (cf. [24, 25, 2] 3 Here and in the following we use the convention that for every number q ∈ (1, ∞) we define the number q := q/(q − 1).

O O O B I O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O