Pricing for Utility-Driven Resource Management and Allocation in Clusters Users perceive varying levels of utility for each different job completed by the cluster. Therefore, there is a need for existing cluster resource management systems (RMS) to provide a means for the user to express its perceived utility during job submission. The cluster RMS can then obtain and consider these user-centric needs such as Qualityof-Service requirements in order to achieve utility-driven resource management and allocation. We advocate the use of computational economy for this purpose. In this paper, we describe an architectural framework for a utility-driven cluster RMS. We present a user-level job submission specification for soliciting user-centric information that is used by the cluster RMS for making better resource allocation decisions. In addition, we propose a dynamic pricing function that the cluster owner can use to determine the level of sharing within a cluster. Finally, we define two user-centric performance evaluation metrics: Job QoS Satisfaction and Cluster Profitability for measuring the effectiveness of the proposed pricing function in realizing utility-driven resource management and allocation. Cluster computing (Pfister 1998; Buyya 1999 ) is increasingly used for high-performance, high-throughput and high-availability computing in a wide variety of application areas. Clusters are not only used for executing compute-intensive applications, but also as replicated storage and backup servers that provide the essential fault tolerance and reliability for critical applications.. Mission critical cluster middlewares create a single system image (SSI) (Buyya, Cortes and Jin 2001) that presents a single unified computing resource to the user. This provides better usability and transparency for the users as it hides the complexities of the underlying distributed and heterogeneous nature of clusters from them. An example of such a middleware is the cluster resource management system (RMS) that provides a single interface for user-level sequential and parallel applications to be executed on the cluster.. For effective and efficient management, the cluster RMS requires knowledge of how users value the resources that are being competed for (Buyya, Abramson and Giddy 2001) and having a feedback mechanism that prevents users from submitting unlimited quantities of work (Chun and Culler 2002) . However, existing cluster RMSs provide no or minimal support for users to define Quality of Service (QoS) requirements during job submission. For instance, the user cannot specify the deadline when the job should finish execution and the budget that he is willing to pay for the execution before the deadline. They continue to use system-centric approaches that focus on increasing the throughput and maximizing the utilization of the cluster. They neglect the need to use utility models for allocation and management of resources that would otherwise consider, and thus be able to achieve, the users' desired utility.. We advocate the use of computational economy (Waldspurger et al. 1992; Stonebraker et al. 1994; Culler 2000a, 2000b; Sherwani et al. 2004 ) for achieving utility-driven resource management and allocation in clusters since system-centric management for shared resources is not effective due to lack of economic accountability. Computational economy is able to regulate supply and demand of cluster resources at market equilibrium, provides feedback in terms of economic incentives for both users and cluster owner, and promotes QoS-based resource allocation that caters to users' needs.. This paper focuses on a pricing mechanism to support utility-driven management and allocation of resources in a cluster. First, the architecture of existing cluster RMS that uses system-centric approaches is extended to adopt economy-based resource management and allocation. A simple and extensible user-level job submission specification provides a means for users to specify user-centric information such as resource and QoS requirements. Economy-based mechanisms then make use of this information and incorporate a pricing function to enforce resource allocations. The effectiveness of the economybased mechanisms is examined using two user-centric evaluation metrics: Job QoS Satisfaction and Cluster Profitability.. The rest of this paper is organized as follows. Section 2 discusses related work. Section 3 presents an architectural framework for a utility-driven cluster RMS. Section 4 describes the user-level job submission specification for soliciting user-centric information for each job. Section 5 defines a pricing function that satisfies four essential requirements for pricing cluster resources. Section 6 outlines the admission control, resource allocation, and job control mechanisms that together enforce the utility to be achieved by the cluster. Section 7 discusses performance evaluation results of the proposed pricing function using two user-centric evaluation metrics and Section 8 concludes this paper. We have demonstrated the importance of an effective pricing mechanism for achieving utility-driven resource management and allocation in clusters, especially when demand exceeds supply of cluster resources. We show that our enhanced pricing function meets the four essential requirements for pricing of resources. In particular, our pricing function provides flexibility for the cluster owner to easily configure the pricing of his cluster resources to modify the level of sharing. Our pricing function also adapts to the changing supply and demand of resources by computing higher pricing when cluster workload increases. This serves as an effective means for admission control to prevent the cluster from overloading and to tolerate job runtime underestimates. Finally, the pricing function generates a higher economic benefit for the cluster owner. There are a number of cluster RMSs such as Condor , and Sun Grid Engine (SGE) (Sun Microsystems 2002) . But, these existing Cluster RMSs adopt systemcentric approaches that optimize overall cluster performance. For example, the cluster RMS aims to maximize processor throughput and utilization for the cluster, and minimize average waiting time and response time for the jobs. But, these system-centric approaches neglect and thus ignore user-centric required services that truly determine users' needs and utility. There are no or minimal means for users to define QoS requirements and their valuations during job submission so that the cluster RMS can improve the value of utility. We propose an architectural framework for extending these existing cluster RMSs to support utility-driven resource management and allocation, and describe how economy-based mechanisms can be incorporated to achieve that.. Maui (Supercluster Research and Development Group 2004) is an advanced scheduler that supports configurable job prioritization, fairness policies and scheduling policies to maximize resource utilization and minimize job response time. It provides extensive options for the administrator to configure and define various priorities of jobs to determine how resources are allocated to jobs. Maui also allows users to define QoS parameters for jobs that will then be granted additional privileges and supports advance reservation of resources whereby a set of resources can be reserved for specific jobs at a particular timeframe. In addition, Maui can be integrated as the scheduler for traditional cluster RMS such as LoadLeveler, LSF, PBS and SGE. However, Maui does not provide economic incentives for users to submit jobs with lower priority or QoS requirements and cluster owners to share resources.. REXEC (Chun and Culler 2000b ) is a remote execution environment for a campus-wide network of workstations, which forms part of the Berkeley Millennium Project. REXEC allows the user to specify the maximum rate (credits per minute) that he is willing to pay for processor time. The REXEC client then selects a compute node that matches the user requirements and executes the application directly on it. It uses a proportional resource allocation mechanism that allocates resources to jobs proportional to the user valuation irrespective of their job needs. However, our economy-based resource allocation mechanism prioritizes and allocates resources to jobs based on the QoS needs of each job. We allocate resources proportionally to jobs with respect to their required QoS such as deadline rather than user valuation so that more jobs are completed with their QoS fulfilled.. Libra (Sherwani et al. 2004 ) is an initial project that successfully demonstrates that an economy-based scheduler is able to deliver more utility to users compared to traditional scheduling policies. Libra allows users to specify QoS requirements and allocates resources to jobs proportional to their specified QoS requirements. Thus, Libra can complete more jobs with their QoS requirements satisfied as compared to system-centric scheduling policies that do not consider various QoS needs of different jobs. Currently, Libra computes a static cost that provides incentives for jobs with a more relaxed deadline so as to encourage users to submit jobs with a longer deadline. However, Libra does not consider the actual supply and demand of resources, thus users can continue to submit an unlimited amount of jobs into the cluster if they have the budget. In this paper, we propose an enhanced pricing function that satisfies four essential requirements for pricing of cluster resources and prevents the cluster from overloading.

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Evaluation of four models for the sonification of elite rowing Abstract Many aspects of sonification represent potential benefits for the practice of sports. Taking advantage of the characteristics of auditory perception, interactive sonification offers promising opportunities for enhancing the training of athletes. The efficient learning and memorizing abilities pertaining to the sense of hearing, together with the strong coupling between auditory and sensorimotor systems, make the use of sound a natural field of investigation in quest of efficiency optimization in individual sports at a high level. This study presents an application of sonification to elite rowing, introducing and evaluating four sonification models. The rapid development of mobile technology capable of efficiently handling numerical information offers new possibilities for interactive auditory display. Thus, these models have been developed under the specific constraints of a mobile platform, from data acquisition to the generation of a meaningful sound feedback. In order to evaluate the models, two listening experiments have then been carried out with elite rowers. Results show a good ability of the participants to efficiently extract basic characteristics of the sonified data, even in a non-interactive context. Qualitative assessment of the models highlights the need for a balance between function and aesthetics in interactive sonification design. Consequently, particular attention on usability is required for future displays to become widespread. We presented an advanced quantitative evaluation of four different models for the sonification of rowing. The lack of advanced methods for the evaluation of sonification systems is a critical issue for the field of auditory display. Few quantitative approaches enabling an objective comparison of different sonification paradigms have been used in the past, as it was shown by Vogt [29] in a review of evaluation works published within the International Community for Auditory Display (ICAD) between 1992 and 2009.. In the present study, it was revealed that at the current state of development, our sonification models allow to differentiate simple characteristics of the data (strong, fast) but not to extract more advanced information (gender, experience). The listeners were able to fairly estimate the stroke rate of randomized acceleration samples, notwithstanding a bias in the estimation of its absolute value. The estimation of the stroke rate did not depend on the sonification model that was used. Nevertheless, a greater variety of numerical values for the stroke rate should be used in future experiments in order to confirm these results. It was found that rowers correlated the amount of information that they assumed to be able to extract from a particular sonification model to its aesthetic appreciation. Despite the personal nature of sound experience, a ranking could be established with respect to both aesthetic and overall preferences, the latter being rather influenced by aesthetic aspects than by functional aspects. However, marked differences were found in the acceptance rate of specific sonification models between two subcategories (elite rowers and casual rowers), while the overall acceptance rate of the principle of sonification was comparable. Finally, the nature of the experimental protocol was found to have a major influence on ratings and acceptance rate of the sonification models.. Future work should include on-water experiments to assess the interactivity of the system when the sonification works in real-time. The development of further models of advanced complexity appears to be necessary, in order to provide more options of sound display to the rowers, as they experience the sonification models in a very personal way. The existing models could be modified, keeping in mind that the aesthetic quality of any sound display should be taken into serious consideration. For example, specific scales could be used in Musical instruments instead of a chromatic ascent. More sensors, for example biofeedback probes, should be used in the data collection so that the athletes would be able to extract more advanced information from the sound. Finally, there is a need for a general reflexion to determine which specific sound parameters can be coupled to particular physical quantities in a relevant manner. Questions about characteristics of the sound stimuli were asked in order to assess the ability of the participants to extract information from the sonification models. One acceleration sample corresponded to a beginner rower and all the others to athletes from the Swedish national team. In both experiments, the participants succeeded very well to spot the sample of the beginner, assessing the resulting sounds as corresponding to a weaker and slower rowing technique. However, the subjects failed to associate this information to the inexperience of that particular rower. This might indicate that they didn't believe to be able to extract advanced properties such as the gender and the experience of the rower from models being based solely on one kinematic quantity as input parameter: no significant differences were found be- tween the answers to these questions for the factor acceleration sample. The four models led to comparable results in information extraction. The Car engine model was rated as more "masculine" and the Wind model was rated as more "senior". However, the effect size was found respectively small and medium for these two significant differences.. Although not matching perfectly the actual numerical values, estimation of the stroke rate revealed a clear separation between the two groups of acceleration samples corresponding to 17-18 and 26 strokes per minute, as shown in Tables 6 and 7 . No significant differences were found between the models, which shows that more complex sounds did not alter the extraction of this particular information by the participants.. Answers to questions concerning individual preferences towards the models were all significantly correlated with each other. According to our expectations, larger correlations were found within two categories (function, aesthetics). In addition, significant correlations also appeared between elements of these two subsets. This indicates that rowers associated to some extent the aesthetic quality of a model to the amount of information they assumed to be able to extract from it.. Only a few significant differences were found in the qualitative ratings of the models, which underlines the personal character of the experience of sound and therefore illustrates the importance of having various sound models available for the athletes to choose among. However, the analysis of mean scores enabled to observe that Pure tone was considered the least functional model, and to establish a ranking with respect to aesthetic ratings: 1. Wind, 2. Musical instruments, 3. Pure tone, 4. Car engine. Explicit ranking and acceptance rate led to similar rankings, with the exception that Pure tone was ranked lower than Car engine in both cases. Altogether, this indicates that the participants ranked the models according to the aesthetic value it was awarded rather than according to how informative they thought the model was, although these two properties have been shown to be interrelated. Nevertheless, a presumed poor functionality-as for Pure tone-proved to have a certain influence on the overall acceptance of the system.. As mentioned in Sect. 3.2.4, two models using a similar sound material as our models were assessed in [1] : sinification and weather metaphor, corresponding respectively to Pure tone and Wind. It is therefore interesting to compare the results obtained by these two pairs of models. In the present work, Wind was clearly the most preferred model, receiving the highest ratings both with respect to aesthetic qualities and by overall judgment (explicit ranking and acceptance rate). By contrast, weather metaphor was the least preferred models of [1] in terms of time of use, which was considered as the main criterion for evaluation. Nevertheless, it was shown in post-trial interviews that it was well appreciated, being mentioned as the "most liked" model by more subjects than as the "least liked" one. In the present work, Pure tone was our least popular model, ranking as penultimate with respect to aesthetic qualities, and as last with respect to functionality and by overall judgment. By contrast, sinification was the second most preferred model in terms of time of use. Post-trial interviews showed that it was mentioned as the "most liked" and as the "least liked" model by a comparable number of subjects. These discrepancies between the two studies can be explained by several factors: the couplings were not exactly the same (using acceleration as input parameter while we chose to display velocity in the corresponding models), the experimental conditions were very different (interactive testing in the context of recreational sports, while we performed listening tests in the context of competitive training), and so was the user popula-tion as well (HCI researchers in [1] , elite and casual rowers in our study).. Remarkably, the experimental protocol seemed to strongly influence the overall ratings of the sonification models, the participants showing less leniency in the more impersonal mode of an online survey. This is reflected by the significant difference in the proportion of rowers willing to use the models (57.1% in the pilot study, 30.3% in the second experiment). A possible influential factor could be the fact that rowers of international level are more disposed to accept annoying additions to their training provided that it would improve the performance, but the separation into two subcategories (elite rowers, casual rowers) in the second experiment yields to comparable overall acceptance percentages (cf. Table 9 ), indicating clearly that this is not the case. The fact that Schaffert et al. found even higher acceptance rates [22] [23] [24] suggests that trying the sonification system in real-time conditions before the evaluation could have a significant influence on the degree of acceptance of the models.. A more detailed analysis of the acceptance rate revealed differences in the model preferences between the two subcategories of subjects (elite rowers, casual rowers). This is consistent with the finding-in a different context, namely recreational sports-by Barrass et al. [1] that there exist general trends in preferences, and "subgroups with different aesthetic and functional requirements". In the present experiment, elite rowers seemed to prefer models making use of metaphorical associations (Wind, Car engine) whereas casual rowers showed a better acceptance of "musical" models (Musical instruments, Pure tone). An explanation could be that the models using ecological sounds and metaphors lead to a lower cognitive load than others, which is most certainly crucial when applied in the context of a demanding training. The present study is however limited by the relatively small number of participants, and therefore only allows for speculation in this regard: further studies would be required in order to draw more definitive conclusions.. Informal feedback from rowers during direct conversations or through free comments in the questionnaires revealed a certain interest for the principle of sonification. Some of them came up with ideas for other potential uses of sound for enhancing the training, e.g. to help the synchronization of a crew. On the other hand, they were particularly critical about aesthetics of the sounds. Some stated that they could use one of the models but staunchly rejected all the others. Interestingly, this preferred model was not always the same among the participants. Some pointed out that they could probably not stand a system producing sound continuously but that a discrete feedback as the one used in Musical instruments would be useful. Several rowers suggested to use synthesized water sounds, e.g. splashing and flowing water. Interestingly, this idea was implemented in the ecological sonification model by Williamson and Murray-Smith cited in [22] but not further evaluated, to our knowledge. To evaluate and compare the four models, two listening tests were conducted. The first experiment included only two models and served as a pilot study. It was performed at the Bosön sport technology center in Lidingö, Sweden with rowers from the Swedish national team. The second experiment was conducted as an online survey in order to collect answers from a larger panel of athletes. It included the four sonification models. Questionnaires were set up in order to assess the extent of information transmitted by the sound in offline conditions, as well as the preferences of the participants with respect to both function and aesthetics.

O O O O O O B O O B O O O O B O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Mobile Starfield Visualization with Space Compression Capabilities In this paper, we present MoStarD, a mobile application that couples starfield displays with dynamic queries to support user exploration of large datasets. A novel space compression technique enables users to compress areas of the starfield display to optimize screen space usage and easily focus on specific subsets of a dataset. Starfield displays combine 2D scatterplots for visual information presentation with additional features to support selection and zooming [1] . When coupled with dynamic queries, starfield displays are a powerful tool to explore datasets of multi-attribute elements, providing users with a fast and easy-to-use method to specify queries and visually analyze results.. Despite the potential benefits, very few attempts have been made at adapting starfield displays and dynamic queries to the mobile domain. One of the first mobile implementations of these techniques [4] visualized a small movie database on a palmtop and allowed users to filter movies by genre and obtain details by tapping single movie elements. In [3] , this approach has been enhanced by providing comprehensive panning and zooming capabilities as well as enabling users to change the variable associated to each axis. A subsequent work [2] compared two alternative interaction techniques to prune visual clutter in a starfield display and to make each item easily accessible to the user.. In this paper, we describe MoStarD (Mobile Starfield Display), a flexible mobile visualization and data exploration solution based on starfield displays and dynamic queries that integrates a novel space compression technique to optimize screen space usage. This paper showed how it is possible to couple starfield displays with dynamic queries on mobile devices. The proposed approach allows users to look at the effect of queries in real-time while manipulating query devices. The novel space compression technique we introduced increases user flexibility in exploring a dataset and further optimizes screen space usage. MoStarD usability has been informally checked on a few users with positive results. We now plan to carry out a comprehensive evaluation, also comparing MoStarD with the starfield displays for mobile devices described in Section 1.

O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O


Out-of-core compression for gigantic polygon meshes Polygonal models acquired with emerging 3D scanning technology or from large scale CAD applications easily reach sizes of several gigabytes and do not fit in the address space of common 32-bit desktop PCs. In this paper we propose an out-of-core mesh compression technique that converts such gigantic meshes into a streamable, highly compressed representation. During decompression only a small portion of the mesh needs to be kept in memory at any time. As full connectivity information is available along the decompression boundaries, this provides seamless mesh access for incremental in-core processing on gigantic meshes. Decompression speeds are CPU-limited and exceed one million vertices and two million triangles per second on a 1.8 GHz Athlon processor.A novel external memory data structure provides our compression engine with transparent access to arbitrary large meshes. This out-of-core mesh was designed to accommodate the access pattern of our region-growing based compressor, which -in return -performs mesh queries as seldom and as local as possible by remembering previous queries as long as needed and by adapting its traversal slightly. The achieved compression rates are state-of-the-art. The standard representation of a polygon mesh uses an array of floats to specify the vertex positions and an array of integers containing indices into the vertex array to specify the polygons. For large and detailed models this representation results in files of gigantic size that consume large amount of storage space. The St. Matthew model from Stanford's Digital Michelangelo Project [Levoy et al. 2000] , for example, has over 186 million vertices resulting in more than six gigabytes of data. Transmission of such gigantic models over the Internet consumes hours and even loading them from the hard drive takes tens of minutes.. The need for more compact representations has motivated research on mesh compression and a number of efficient schemes have been proposed [Deering 1995; Taubin and Rossignac 1998; Touma and Gotsman 1998; Gumhold and Strasser 1998; Li and Kuo 1998; Rossignac 1999; Bajaj et al. 1999; Isenburg and Snoeyink 2000; Lee et al. 2002] . Ironically, none of these schemes is capable-at least not on common desktop PCs-to deal with meshes of the giga-byte size that would benefit from compression the most. Current compression algorithms and some of the corresponding decompression algorithms can only be used when connectivity and geometry of the mesh are small enough to reside in main memory. Realizing this limitation, Ho et. al [2001] propose to cut gigantic meshes into manageable pieces and encode each separately using existing techniques. However, partitioning the mesh introduces artificial discontinuities. The special treatment required to deal with these cuts not only lowers compression rates but also significantly reduces decompression speeds.. Up to a certain mesh size, the memory requirements of the compression process could be satisfied using a 64-bit super-computer with vast amounts of main memory. Research labs and industries that create giga-byte sized meshes often have access to such equipment. But to decompress on common desktop PCs, at least the memory foot-print of the decompression process needs to be small. In particular, it must not have memory requirements in the size of the decompressed mesh. This eliminates a number of popular multipass schemes that either need to store the entire mesh for connectivity decompression [Taubin and Rossignac 1998; Rossignac 1999; Bajaj et al. 1999] or that decompress connectivity and geometry in separate passes [Isenburg and Snoeyink 2000; Karni and Gotsman 2000; Szymczak et al. 2002] .. This leaves us with all one-pass coders that can perform decompression in a single, memory-limited pass over the mesh. Such schemes (de-)compress connectivity and geometry information in an interwoven fashion. This allows streaming decompression that can start producing mesh triangles as soon as the first few bytes have been read. There are several schemes that could be implemented as one-pass coders [Touma and Gotsman 1998; Gumhold and Strasser 1998; Li and Kuo 1998; Lee et al. 2002] .. In this paper we show how to compress meshes of giga-byte size in one piece on a standard PC using an external memory data structure that provides transparent access to arbitrary large meshes. Our out-of-core mesh uses a caching strategy that accommodates the access pattern of the compression engine to reduce costly loads of data from disk. Our compressor is based on the scheme of Touma and Gotsman [1998] and uses degree coding and linear prediction coding to achieve state-of-the-art compression rates. It borrows ideas from Isenburg [2002] to adapt the traversal, from Gumhold and Strasser [1998] to handle mesh borders, and from to deal with non-manifold meshes. Our compressed format allows streaming, small memory foot-print decompression at speeds of more than 2 million triangles a second.. The snap-shots in Figure 1 visualize the decompression process on the St. Matthew statue. For steps (a) to (g) we stored every 1000th decompressed vertex in memory, while (h) is an example out-of-core rendering. Using less than 10 MB of memory, this 386 million triangle model loads and decompresses from a 456 MB file off the hard-drive in only 174 seconds. At any time only the green boundaries need to be kept in memory. Decompressed vertices and triangles can be processed immediately, for example, by sending them to the graphics hardware. The out-of-core rendering took 248 seconds to complete with most of the additional time being spent on computing triangle normals. These measurements were taken on a 1.8 Ghz AMD Athlon processor with an Nvidia Geforce 4200 card.. Our compressed format has benefits beyond efficient storage and fast loading. It is a better representation of the raw data for performing out-of-core computations on large meshes. Indexed mesh formats are inefficient to work with and often need to be de-referenced in a costly pre-processing step. The resulting polygon soups are at least twice as big and, although they can be efficiently batchprocessed, provide no connectivity information. Our compressed format streams gigantic meshes through limited memory and provides seamless mesh access along the decompression boundaries, thereby allowing incremental in-core processing on the entire mesh.. The remainder of this paper is organized as follows: The next section summarizes related work on out-of-core processing, out-ofcore data structures, and mesh compression. In Section 3 we introduce our out-of-core mesh and describe how to build it from an indexed mesh. Then, in Section 4, we describe our compression algorithm and report resulting compression rates and decompression speeds on the largest models that were available to us. The last section summarizes our contributions and evaluates their benefits for other algorithms that process gigantic polygon meshes. We presented a technique that is able to compress gigantic models such as the St. Matthew statue in one piece on standard desktop PCs. Our compression rates are about 25% lower and our decompression speeds about 100 times faster than the technique by Ho et al. [2001] that processes such models by cutting them in pieces.. For this, we introduced an external memory data structure, the out-of-core mesh, that provides our compressor with transparent access to large meshes. We described how to efficiently build this data structure from an indexed mesh representation using only limited memory. Our out-of-core mesh may also be useful to other algorithm that process large meshes. To use it efficiently the order of mesh queries should be localized, but most traversal-based processing is readily accommodated. While our current implementation only allows index-pairs to use a combined maximum of 32 bits, this data structure can theoretically handle arbitrary large meshes. Storing more bits per index-pair, however, will increase in-core and on-disk storage and make its build-up/usage more IO-limited.. Our compressed format has benefits beyond efficient storage and fast loading. It provides better access to large meshes than indexed formats or polygon soup by allowing to stream gigantic meshes through limited memory while providing seamless connectivity information along the decompression boundary. This streaming mesh representation offers a new approach to out-of-core mesh processing that combines the efficiency of batch-processing with the advantages of explicit connectivity information as it is available in online-processing. The concept of sequenced processing restricts the access to the mesh to a fixed traversal order, but at the same time provides full connectivity for the active elements of this traversal.. Traversing mesh triangles in a particular order is already used for fast rendering on modern graphics cards. The number of times a vertex needs to be fetched from the main memory is reduced by caching previously received vertices on the card. The triangles are sent to the card in a rendering sequence that tries to minimize cache misses [Hoppe 1999 ]. Misses cannot be avoided altogether due to the fixed size of the cache [Bar-Yehuda and Gotsman 1996] .. In a similar spirit our compressed format provides processing sequences for more efficient mesh processing-but at a much larger scale. With the main memory as the "cache" we usually have more than enough storage space for all active elements throughout the traversal of a mesh. Therefore the analogue of a "cache miss" does not exist. Any algorithm that requires a complete mesh traversal without being particular about its order can perform computations at decompression speed-we can envision an entire breed of mesh processing algorithms adapted to this kind of mesh access. Performance results of the out-of-core mesh are gathered in Table 1 for Lucy, David (1mm), and St. Matthew. The in-core memory was restricted to 96/192/384 MB and we allowed 192/384/768 clusters to be cached simultaneously. The resulting out-of-core meshes consumed 0.8/1.7/11.2 GB on disk with the build times being dominated by the face pass and the inverse half-edge matching. The best compression times are achieved when enough clusters are cached to cover the entire compression boundary. But since its maximal length is not known in advance, this cannot be guaranteed. If too few clusters are cached, the compression process becomes heavily IO-limited. However, even then compression times are acceptable given the small in-core memory usage. Lucy, for example, has a poor cache quality factor Q read of 11.0. Although Q read for Lucy is much better with c cache = 384, handling the larger number of files results in an overall longer running time. Twice the number of clusters as MB of in-core storage seemed a good trade-off between the two. When increasing S incore to 128 MB and caching 256 clusters, then Q read is 2.1 and Lucy compresses in about 5 minutes.. The compression gains of our representation over standard binary PLY are listed in Table 3 . Depending on the chosen level of precision the compression ratios range from 1 : 10 to 1 : 20. Comparing measurements on the same models to Ho et al. [2001] , our bit-rates are about 25% better. Another advantage of our compressed format is the reduced time to load a mesh from disk. Decompression speeds are CPU-limited and exceed one million vertices and two million triangles per second on a 1.8 GHz Athlon processor.. The compression rates for connectivity and for geometry at different precision levels are detailed separately in Table 4 . One immediate observation is that an additional precision of 2 bits increases some geometry compression rates by about 6 bits per vertex (bpv) or more. While predictive coding is known not to scale well with increasing precision, here this is likely a sign for the precision of quantization being higher than that of the data samples. In this case the additional two bits only add incompressible noise to the data. Out-of-core or external memory algorithms that allow to process vast amounts of data with limited main memory are an active research area in visualization and computer graphics. Recently proposed out-of-core methods include isosurface extraction, surface reconstruction, volume visualization, massive model rendering, and-most relevant to our work-simplification of large meshes. Except for the work by Ho et al. [2001] , out-of-core approaches to mesh compression have so far received little attention.. The two main computation paradigms of external memory techniques are batched and online processing: For the first, the data is streamed in one or more passes though the main memory and computations are restricted to the data in memory. For the other, the data is processed through a series of (potentially random) queries. In order to avoid costly disk access with each query (e.g. thrashing) the data is usually re-organized to accommodate the anticipated access pattern. Online processing can be accelerated further by caching or pre-fetching of data that is likely to be queried [Silva et al. 2002] .. Out-Of-Core Simplification methods typically make heavy use of batch-processing. Lindstrom [2000] first creates a vertex clustering [Rossignac and Borrel 1993] in the resolution of the output mesh and stores one quadric error matrix per occupied grid cell in memory. Indexed input meshes are first dereferenced into a polygon-soup and then batch-processed one a triangle at a time by adding its quadric to all cells in which it has a vertex. Later, Lindstrom showed together with Silva [2001] that the limitation of the output mesh having to fit in main memory can be overcome using a series of external sorts.. A different approach for simplifying huge meshes was suggested by Hoppe [1998] and Bernardini et al. [2002] : The input mesh is partitioned into pieces that are small enough to be processed in-core, which are then simplified individually. The partition boundaries are left untouched such that the simplified pieces can be stitched back together seamlessly. While the hierarchical approach of Hoppe automatically simplifies these boundaries at the next level, Bernardini et al. simply process the mesh more than once-each time using a different partitioning.. The methods discussed so far treat large meshes different from small meshes as they try to avoid performing costly online processing on the entire mesh. Therefore the output produced by an out-of-core algorithm is usually of lower quality than that of an incore algorithm. Addressing this issue, Cignoni et. al [2003] propose an octree-based external memory data structure that provides algorithms with transparent online access to huge meshes. This makes it possible to, for example, simplify the St. Matthew statue from 386 to 94 million triangles using iterative edge contraction.. Albeit substantial differences, our out-of-core mesh is motivated by the same idea: it provides the mesh compressor transparent access to the connectivity and geometry of gigantic meshes. Therefore our compressor will produce the same result, no matter if used with our out-of-core mesh or with the entire mesh stored in-core.. Out-Of-Core Data Structures for Meshes have also been investigated by McMains et. al [2001] . They reconstruct complete topology information (e.g. including non-manifoldness) from polygon soups by making efficient use of virtual memory. Their data structure provides much more functionality than our compressor needs and so its storage requirements are high. Also, using virtual memory as a caching strategy would restrict us to 4 GB of data on a PC and we will need more than 11 GB for the St. Matthew statue.. The octree-based external memory mesh of Cignoni et. al [2003] could be adapted to work with our mesh compressor. It has roughly the same build times and stores only slightly more data on disk. However, their octree nodes do not store explicit connectivity information, which has to be built on the fly. While this is acceptable for a small number of loads per node, the query order of our compressor might require to load some nodes more often-especially if we used their octree-based mesh: its nodes are created through regular space partitioning, which is insensitive to the underlying connectivity, while our clusters are compact along the surface.. Mesh Compression techniques have always overlooked the memory requirements of the decompression process. So far meshes were moderately sized and memory usage was at most linear in mesh size. However, today's meshes most in need of compression are those above the 10 million vertex barrier. The memory limitation on common desktop PCs allows the decompression process only a single, memory-limited pass over such meshes. This eliminates all schemes that need to store the entire mesh for connectivity decompression [Taubin and Rossignac 1998; Rossignac 1999; Bajaj et al. 1999] or that decompress connectivity and geometry in separate passes [Isenburg and Snoeyink 2000; Karni and Gotsman 2000; Szymczak et al. 2002] . Naturally, this constraint also prohibits the use of progressive approaches that require random mesh access for refinement operations during decompression Cohen-Or et al. 1999; Pajarola and Rossignac 2000; Alliez and Desbrun 2001a] . And finally, the sheer size of the data prohibits computation-heavy techniques such as traversal optimizations [Kronrod and Gotsman 2002] , vector quantization [Lee and Ko 2000] , or expensive per-vertex computations [Lee et al. 2002] .. This leaves all those methods whose decompressor can be restricted to a single, memory-limited, computation-efficient pass over the mesh. This coincides with all those methods whose compressor can be implemented as a fast one-pass coder [Touma and Gotsman 1998; Gumhold and Strasser 1998; Li and Kuo 1998 ].. All these compression algorithms require access to explicit connectivity information, which is usually constructed in a preprocessing step. However, if the mesh does not fit into main memory already this is not possible. Therefore, Ho et. al [2001] suggest to cut large meshes into smaller pieces that can be dealt with in-core. They process each piece separately by first constructing explicit connectivity, which is then compressed with the two-pass coder of Rossignac [1999] , before compressing the vertex positions with the parallelogram predictor of Touma and Gotsman [1998] in a third pass. They record additional information that specifies how to stitch the pieces back together after decoding.. The compression scheme we propose here has several advantages over that of Ho et. al [2001] . As we do not break up the model, our compression rates are 20 to 30 percent better. As we can decode the entire model in a single pass, our decompression speeds are about 100 times faster. Finally, as our decompressor streams the entire mesh through main memory with a small memory footprint, our compressed representation is useful beyond reduced file sizes and shortened download times. It supports efficient batchprocessing for performing computation on large meshes while at the same time providing seamless access to mesh connectivity.. We should also mention shape compression methods [Khodakovsky et al. 2000; Gu et al. 2002; Szymczak et al. 2002] as they are especially well suited for converting detailed scanned datasets into highly compressed representations. These approaches remesh prior to compression under the assumption that not a particular mesh but rather the geometric shape that it represents is of interest. While these schemes were not designed to handle gigantic meshes, they might benefit from the concepts to out-of-core processing presented here. However, remeshing methods are not applicable to CAD data such as the Double Eagle model (shown in Figure 6 ).

O B O O B O O O O O O B O O O O O O O B O O O O O O O O O B O O O B O O O O O O O B O O O O B B O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B O O O O O O B O O O O O B B O O O O O O O O O B B O O O O O O O O O O B O O O O O O O O O O O O B O O O O O B O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O B B O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O B O O O O O B O O O O B O O O O O O O O O O O O O O B I O B B O O O O O O O B O O O O O O O O O B O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O B B O O O O O O B O O O O O O O O B O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O B B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O B B O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O


Privacy

preserving

schema

and

data

matching

In

many

business

scenarios,

record

matching

is

performed

across

different

data

sources

with

the

aim

of

identifying

common

information

shared

among

these

sources.

However

such

need

is

often

in

contrast

with

privacy

requirements

concerning

the

data

stored

by

the

sources.

In

this

paper,

we

propose

a

protocol

for

record

matching

that

preserves

privacy

both

at

the

data

level

and

at

the

schema

level.

Specifically,

if

two

sources

need

to

identify

their

common

data,

by

running

the

protocol

they

can

compute

the

matching

of

their

datasets

without

sharing

their

data

in

clear

and

only

sharing

the

result

of

the

matching.The

protocol

uses

a

third

party,

and

maps

records

into

a

vector

space

in

order

to

preserve

their

privacy.Experimental

results

show

the

efficiency

of

the

matching

protocol

in

terms

of

precision

and

recall

as

well

as

the

good

computational

performance.

Record

matching

is

the

process

of

identifying

if

two

(or

more)

records

represent

the

same

real

world

entity

or

not.

Within

a

single

source,

record

matching

is

executed

for

identifying

and

eliminating

duplicate

records.

Across

sources,

record

matching

can

be

performed

for

improving

the

data

quality

of

a

source

by

means

of

correction

by

comparisons

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page.

To

copy

otherwise,

to

republish,

to

post

on

servers

or

to

redistribute

to

lists,

requires

prior

specific

permission

and/or

a

fee.

with

a

better

quality

source,

and

also

for

the

purpose

of

integrating

data..

In

this

paper,

we

focus

on

record

matching

performed

with

the

aim

of

identifying

common

information

shared

by

two

data

sources.

In

this

case,

records

referring

to

the

same

entity

can

represent

that

entity

differently

due

to

any

of

the

following

factors:

different

keys

(key-conflicts),

errors

about

the

values

of

the

record

attributes

(attribute-conflicts)

and

possibly

schema-level

inconsistencies,

such

as

different

descriptions

(formats,

types,

etc.),

different

models,

and

different

structural

representations.

Several

applications

may

require

distinct

data

sources

to

exchange

and

share

common

data.

For

instance

consider

an

eGovernment

scenario,

in

which

an

application

wants

to

make

controls

about

individuals

that

are

suspected

of

tax

evasion.

The

application

may

require

the

tax

agency

to

share

data

concerning

suspected

individuals

with

other

databases,

e.g.

a

customer

database

of

a

company

selling

luxury

goods;

a

record

matching

between

the

agency's

database

and

the

company's

one

can

link

the

same

individuals

and

help

the

tax

control

process.

Though

on

one

side

such

applications

need

to

share

and

integrate

data,

on

the

other

side

they

have

to

deal

with

a

major

and

legitimate

issue:

protecting

the

privacy

of

the

individuals

to

whom

such

data

is

related.

In

the

above

example,

the

tax

agency

cannot

simply

send

data

related

to

the

suspected

individuals,

but

it

must

adopt

techniques

for

preserving

their

privacy..

In

this

paper,

we

propose

a

privacy-preserving

protocol

to

perform

record

matching

across

two

data

sources,

which

is

more

efficient

than

protocols

based

on

cryptographic

techniques

such

as

privacy-preserving

set

intersection.

Our

protocol

has

an

interesting

new

feature

that

concerns

the

privacy

preservation

of

database

schemas.

Whereas

the

problem

of

privacy

protection

of

data

has

been

investigated,

much

less

attention

has

been

devoted

to

schema

level

information.

If

two

companies

decide

to

match

data

concerning

their

respective

customers,

they

first

have

to

agree

on

a

common

schema

format

to

use

for

the

actual

data

matching.

However,

this

means

that

they

must

reveal

to

each

other

how

they

store

their

own

customer

information.

In

many

cases,

companies

may

not

be

willing

to

give

away

a

competitive

advantage

by

revealing

such

information..

Our

protocol

performs

record

matching

by

using

an

approach

that

preserves

privacy

of

both

data

and

schema

and

has

two

major

innovations

with

respect

to

previous

approaches.

With

respect

to

approaches

dealing

with

private

data

sharing,

such

as

the

one

proposed

by

Agrawal

et

al.

that

only

perform

exact

matching,

our

protocol

performs

privacy-preserving

approximate

matching.

It

is

important

to

notice

that

when

data

across

different

sources

have

heterogeneous

quality,

an

exact

match

may

not

be

very

successful,

as

it

may

likely

result

in

very

few

matches.

In

such

cases,

approximate

match

is

the

only

viable

approach.

Approximate

matching

is

however

an

indirect,

somewhat

complicated

process,

as

it

requires

the

computation

of

distance

functions

among

records,

the

values

of

which

have

to

be

kept

private.

As

an

example,

computing

a

distance

where

and

are

two

data

items

owned

respectively

by

sources

P

and

Q,

requires

both

values

to

be

available

at

the

same

time

to

one

party.

However,

under

privacy

constraints,

such

simple

condition

cannot

be

met

in

that

P

cannot

see

and

Q

cannot

see

Even

if

a

third

neutral

party

is

introduced,

such

party

can

compute

neither

the

distance

between

the

plain

values

of

and

nor

the

distance

between

encrypted

values,

because

encryption

functions

do

not

generally

preserve

similarity

distances.

The

second

major

innovation

is

related

to

the

awareness

of

schema

information.

Beyond

preserving

schema

privacy,

we

exploit

schema

information

throughout

the

instance

matching

process.

This

is

a

major

enhancement

provided

by

our

protocol

when

compared

to

approaches

based

on

secure

set

intersection,

such

as

the

approach

by

Naor

et

al.

These

approaches

typically

consider

only

sets

of

simple

objects,

such

as

strings

or

points.

Therefore

the

comparison

among

these

objects

is

typically

straightforward.

By

contrast

our

approach

considers

sets

of

records

that

have:

(i)

an

intensional

structure

that

deeply

affects

the

instance

matching

protocol

and

(ii)

schema

level

heterogeneities.

Another

important

characteristic

of

our

approach

is

that,

unlike

those

protocols,

it

is

efficient

also

for

large

databases.

For

instance

the

secure

set

intersection

protocol

by

Naor

et

al.

has

a

complexity

in

O(n

where

n

is

the

database

size,

thus

making

this

protocol

not

applicable

to

databases.

Furthermore,

our

technique

does

not

rely

on

the

usage

of

complex

cryptographic

processes,

but

assures

privacy

by

application

of

a

method

widely

used

for

similarity

based

searching

of

complex

objects.

The

main

idea

of

our

work

is

to

embed

records

to

be

matched

in

an

Euclidean

space,

that

is,

a

vector

space

having

the

Euclidean

distance

as

norm,

and

to

perform

the

comparison

in

such

a

space.

The

embedding

method

we

use

assures

privacy

in

that

it

is

based

on

a

random

selections

of

the

axes

space.

The

usage

of

a

metric

space

is

typically

made

when

comparing

images,

documents,

and

complex

objects

for

which

performing

comparison

in

the

original

space

is

time

consuming.

Jin

et

al.

have

used

such

an

approach

applied

to

records

rather

than

to

complex

objects

and

shown

its

efficiency

and

flexibility

with

respect

to

the

use

of

several

similarity

functions.

However,

because

this

proposal

does

not

preserve

privacy,

we

adopt

SparseMap

a

different

embedding

method

which

provides

privacy

guarantees

and

has

important

features

distinguishing

it

from

FastMap

the

embedding

method

used

by

Jin

et

al.

The

rest

of

the

paper

is

organized

as

follows.

Section

defines

the

addressed

problem.

Sections

and

provide

the

detail

of

the

instance

and

schema

matching

protocols,

which

are

combined

in

a

single

one

in

Section

Section

analyzes

the

security

of

the

protocol

and

Section

shows

its

experimental

evaluation.

The

discussion

of

related

work

(Section

and

future

work

(Section

concludes

the

paper.

The

paper

describes

a

protocol

for

privacy-preserving

record

matching

between

two

parties

that

can

have

different

schemas

and

privacy

requirements

also

at

schema

level.

The

protocol

does

not

rely

on

complex

cryptographic

techniques

that

are

proven

to

be

inefficient.

Instead,

it

exploits

the

novel

idea

of

obtaining

privacy

by

embedding

the

records

of

each

of

the

two

parties

in

a

vector

space.

We

have

proved

the

security

of

the

protocol

by

a

detailed

analytical

analysis

and

its

effectiveness

and

efficiency

by

means

of

experimental

validation.

The

experimental

evaluation

has

shown

the

possibility

to

reach

an

almost

recall,

and

very

high

levels

of

precision

that

are

comparable

with

the

ones

achieved

when

performing

record

matching

in

the

original

space.

Moreover,

the

time

performance

experiments

have

shown

the

superiority

of

our

protocol

when

compared

to

methods

that

rely

on

cryptography

for

privacy

preservation.

We

believe

that

our

protocol

could

pave

the

way

for

more

efficient

secure

set

intersection

protocols

for

database

applications..

Future

work

will

address

more

complex

data

integration

scenarios.

First,

we

will

consider

scenarios

with

more

than

two

parties.

In

this

extension,

there

are

several

issues

that

should

be

carefully

designed

and

evaluated;

for

instance,

how

to

optimize

the

step

of

building

the

embedding

space.

Various

kind

of

assumptions

can

also

be

made

on

the

behavior

of

the

parties

involved

in

the

protocol,

such

as

introducing

possible

malicious

behaviors

by

some

of

the

parties.

Moreover,

we

will

investigate

the

possibility

of

using

embedding

methods

different

from

SparseMap

in

order

to

evaluate

their

suitability

for

privacy

preserving

record

matching,

and

compare

them

with

the

results

obtained

with

SparseMap

(for

instance,

in

terms

of

the

complexity

for

building

the

embedded

space

and

the

resulting

quality

of

the

space

itself).

Finally,

an

important

feature

of

our

protocol

is

that

it

can

be

applied

to

database

with

novel

data

types

like

images,

DNA

sequences,

proteins,

etc.

We

plan

to

investigate

the

actual

performance

of

our

protocol

when

applied

to

these

data

types.

The

record

matching

problem

(also

known

as

record

linkage)

has

been

studied

for

more

than

five

decades

(see

the

recent

survey

However,

few

methods

for

private

record

matching

have

been

investigated.

Some

initial

approaches

are

motivated

by

the

strict

privacy

requirements

of

eHealth

applications

The

work

which

is

closely

related

to

ours

is

by

Al

Lawati

et

al.

that

propose

a

third

party

based

protocol.

Their

solution

is

based

on:

(i)

the

usage

of

the

TFIDF

(term

frequency-inverse

term

frequency)

comparison

function,

by

reducing

it

to

a

scalar

product

of

vectors

which

can

be

computed

separately

by

each

party;

(ii)

a

secure

blocking

scheme.

The

approach

has

the

disadvantage

to

work

only

for

a

specific

comparison

function;

also,

as

the

focus

is

mainly

on

efficiency,

the

effectiveness

of

the

approach

has

not

been

assessed.

Conversely,

our

approach

can

be

used

with

different

comparison

functions,

it

has

a

proved

effectiveness

and

guarantees

privacy

preservation

also

at

schema

level..

In

addition

to

the

above

approaches,

there

are

three

major

areas

that

are

closely

related

to

our

work,

even

though

no

work

in

such

areas

addresses

our

exact

problem:

secure

set

intersection,

secure

string

comparison,

private

data

sharing.

Several

approaches

have

investigated

the

secure

set

intersection

problem

(see

and

for

a

survey).

Secure

set

intersection

methods

deal

with

exact

matching

and

are

too

expensive

to

be

applied

to

large

databases

due

their

reliance

on

cryptography.

Furthermore,

these

protocols

deal

with

the

intersection

of

sets

of

simple

elements,

and

are

not

designed

for

exploiting

the

semantics

behind

database

records.

The

latter

is

an

important

feature

of

our

protocols.

The

problem

of

securely

comparing

strings

has

been

addressed

by

homomorphic

encryption

schemes,

characterized

by

the

property

that

E(a)

E(b)

E(a+b).

For

example

Atallah

et

al.

propose

an

algorithm

for

comparing

sequences

based

on

such

schemes.

While

such

algorithm

works

for

sequence

similarity,

like

DNA

sequence

comparison,

the

communication

cost

of

this

algorithm,

proportional

to

the

product

of

the

sequence

lengths,

is

prohibitive

for

databases,

in

that

it

would

require

for

a

database

of

n

records

to

apply

the

protocol

to

compare

each

attribute

value

of

each

of

the

n

records.

Ravimukar

et

al.

propose

a

method

to

compute

distance

metrics

securely,

when

they

can

be

expressed

in

terms

of

weights

to

be

computed

separately

by

the

parties,

like

TFIDF.

Because

such

solution

relies

on

cryptographic

techniques,

it

is

not

efficient

and

appropriate

for

databases.

Agrawal

et

al.

formalize

a

general

notion

of

private

information

sharing

across

databases.

Such

notion

relies

on

commutative

encryption

techniques

and

has

opened

the

way

to

other

related

protocols

These

approaches,

however,

do

not

deal

with

approximate

matching

in

that

they

focus

on

exact

string

matching

for

the

purpose

of

query

answering..

In

addition,

in

Section

we

have

proved

the

superior

time

performance

of

our

data

matching

protocol

when

compared

to

Agrawal

et

al.

In

there

is

the

investigation

of

possible

malicious

behaviors

by

the

parties

involved

in

an

information

sharing

setting

in

which

only

necessary

information

must

be

disclosed.

From

the

data

management

area

in

the

data

mining

community,

there

are

several

contributions

regarding

how

to

preserve

privacy

in

distributed

contexts

such

works

are

mainly

based

on

perturbing

the

original

data

and

at

the

same

time

achieving

correct

data

mining

results.

We

performed

an

extensive

experimental

evaluation

of

our

protocol,

with

three

distinct

goals:.

Finding

the

parameters

necessary

for

building

the

embedded

space

and

testing

the

quality

of

the

embedding.

We

experimentally

derived

several

embedding

parameters

and

we

analyzed

the

distortion

of

the

embedding

resulting

from

the

modified

version

of

the

SparseMap

method

that

we

used..

Illustrating

the

effectiveness

of

the

data

matching

protocol.

We

analyzed

the

effectiveness

of

the

record

matching

in

terms

of

the

classical

recall

and

precision

metrics

and

we

compared

the

obtained

results

with

respect

to

the

same

record

matching

performed

in

the

original

space..

Measuring

the

efficiency

of

the

data

matching

protocol.

We

compared

our

method

to

the

time

performance

of

the

secure

protocol

proposed

by

Agrawal

et

al.

which,

differently

from

our

method,

performs

exact

matching.

Finally,

we

also

show

the

time

performance

of

our

method

when

compared

to

the

record

matching

performed

in

the

original

space..

We

used

three

real

datasets

to

evaluate

our

protocol..

The

first

dataset,

referred

to

as

dataset

represents

a

British

Columbia

voters'

list

(available

at

From

such

data

set,

we

extracted

first

and

last

names,

and

generated

two

data

sets

of

records

each.

The

dataset

contains

roughly

duplicate

records

(errors

in

these

duplicates

were

introduced

synthetically).

The

second

dataset,

referred

to

as

dataset

contains

personal

data

by

a

public

administration.

It

consists

of

two

tables

of

sizes

and

respectively,

with

duplicates

already

identified.

The

fields

used

for

the

experiments

are

first

name,

last

name,

and

the

year

of

birth

of

individuals.

The

third

dataset,

referred

to

as

dataset

contains

information

about

businesses

and

is

owned

by

a

public

administration.

The

size

of

this

data

set

is

about

records.

The

duplicates

of

this

data

set

were

artificially

generated

as

it

has

been

mainly

used

to

test

the

efficiency

of

the

method,

by

extracting

from

it

several

data

samples

of

different

sizes..

We

conducted

out

experiments

on

a

Pentium

Processor

running

Windows

XP,

with

a

RAM

of

We

chose

Java

to

implement

the

experiments,

and

to

leverage

open

source

libraries

available

online.

In

particular,

we

used

Secondstring

library

(available

at

that

provides

a

variety

of

similarity

metrics..

For

indexing

the

embedded

space

we

used

a

KDTree

implementation

(available

at

We

also

used

Java

Cryptography

Extension

(available

at

for

implementing

the

exact

private

matching

protocol

described

in

Agrawal

et

al.

B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An agency theory model of ERP implementation Enterprise resource planning (ERP) systems are becoming rapidly indispensable in order for large and medium sized organizations to run their operations. Therefore, management needs to know the factors that drive successful ERP implementation, a product of the continuous interaction between the implementation consultants and client firms. Agency theory has been successfully used by different researchers to explain relationships between two parties seeking a common outcome. This paper develops a model of testable propositions for applying agency theory to study the relationship between implementation consultants and client organizations deploying the ERP systems, and to consequently evaluate how the relationship affects the implementation success. Top management views information systems outsourcing as an attractive option because IS vendors can achieve specialization and economies of scale in the business of processing information [14] The economies of scale, as well as their past experience, enable vendors to offer cheaper and more effective solutions than those the individual clients could have achieved in-house.. The increased popularity of enterprise resource planning (ERP) systems and of outsourcing services offered by ERP vendors has made ERP outsourcing an attractive option for many organizations. Although an outsourcing contract offers attractive opportunities, vendor failure can result in serious adverse shortand long-term results for the client [14] . In extreme cases, clients have been forced into bankruptcy, unable to tackle the financial consequences generated from outsourcing failure. This is especially true in the case of ERP system implementation projects [15, 20, 50, 51] . For example, Foxmeyer went bankrupt in 1996 after three years of unsuccessful implementation of SAP, suing Arthur Anderson and SAP's U.S. subsidiary and in the process [1, 43] . This illustrates what can happen in large organizations when implementations go wrong in outsourcing situations.. Agency theory describes the environment within a firm or between a set of firms in terms of sets of contracts in which one party (the principal) engages another party (the agent) to perform a service on the principal's behalf which involves delegating part of the decision making authority to the agent [32, 39] . Several studies have applied agency theory to study both general project success and IS project success in principal-agent settings in which one group of people have delegated the responsibility of project implementation to another group [32, 31, 69] .. Vendor relationship management is extremely important for the client to achieve both short-and long-term ERP project success. This study attempts to develop a model that will apply agency theory to explain the vendor-client relationship in ERP outsourcing. Incentives and information affect the implied contractual relationships that exist between principals and agents, namely how principals select agents and how the selected agents subsequently behave [69] . The problem in agency relationships arises because (a) the principal and the agent have different goals and (b) the principal cannot determine if the agent has behaved appropriately [24] . Agency theory researchers refer to this problem as "opportunism" by the agent [24] and have been most concerned with describing the governance mechanism that solves this problem [38] .. The first of the two propositions of agency theory, as described by Eisenhardt [24] , says that agent opportunism can be curbed by incentive alignment between the principal and the agent. The second says that agent opportunism can also be curbed by Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. information systems that enable the principal to verify agent's behavior.. Two kinds of opportunism are cited in the agency theory literature. Known as adverse selection and moral hazard, they form two important aspects of the agency problem. Adverse selection refers to misrepresentation of ability by the agent. The agent may claim to have certain skills or abilities when he or she is hired. However, the principal cannot completely verify these skills and abilities either at the time of hiring or while the agent is working [24] . Moral hazard refers to shirking by the agent where the agent does not put forth the agreed upon effort [24] .. ERP implementations are socially complex activities calling forth coordination of many firms involved [50] . The coordination between the implementation consultants and the client deserves special attention because several adopters have reported having conflicts (sometimes severe) with IT products and service vendors especially over contractual provisions (e.g., pricing and billing arrangement) and project directions (e.g., project management) [50] . Contracting out of information systems to a vendor or consultant is the principal-agent relationship at its clearest, with a written contract binding the parties [59] . The ERP clientconsultant contract forms a subset of that relationship and hence may be appropriately analyzed from an agency theory perspective. With the consultant as agent and the client as principal, typical agency problems are expected to be evident in their relationship.. Tuttle et al. [69] have already established that IS professionals tend to implement a project with quality problems in a moral hazard situation. They conducted a decision-making experiment to examine the effects of moral hazard on information systems professionals' decisions whether or not to knowingly implement a system with quality problems in their own organization. The top management of the organization acted as the principals and the IS professionals played the role of agents in this study. The research findings suggest that professionals who experience the conditions for moral hazard will exhibit a greater tendency to implement a system with quality problems than those who do not face a moral hazard situation. The IS implementation professionals in their study were the employees of the organizations in which the systems were being implemented.. However, the propositions developed in this study will apply agency theory to study moral hazard in a "contracting out" situation where the principal and agent do not belong to the same organization. Also, they will examine adverse selection in the same context.

O B I O O O B B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O