An Ontology for Web Service Ratings and Reputations 

ABSTRACT

E. Michael Maximilien

Munindar P. Singh

IBM Corporation and NCSU

North Carolina State University

maxim@us.ibm.com

mpsingh@csc.ncsu.edu

2. BRIEF OVERVIEW OF ARCHITECTURE

Any conceptual model must rely upon an execution architecture.
Current Web services standards enable publishing service descripFor concreteness, we describe our recently introduced proxy-based
tions and finding services by matching requested and publishedarchitecture [Maximilien and Singh, 2002b]. Our conceptual model
descriptions based on syntactic criteria such as method signatures
is compatible with other architectures as well, but we lack the spaceor service category. Emerging approaches such as DAML-S use
to describe additional candidates here.
DAML to formalize richer models for expressing capabilities of
We propose the addition of a Web Service Agent Proxy (WSAP)
services. DAML-S would go beyond WSDL in terms of the richto access each service. For our purposes, an agent is a software
ness of service descriptions. However, neither current Web services
component that automates some tasks for its principal. Agentsstandards nor DAML-S provide a basis for selecting a good service
communicate with other agents, accept requests from their users,or for comparing services that implement the same interface.
In our view, service selection is the key problem to enable application- and are typically autonomous. A WSAP is an agent that acts as aproxy for clients of Web services. That is, a client would have a
to-application integration, which is the essential vision behind Web
proxy agent for each service that it needs. The agents are knowlservices. Existing approaches don’t address service selection, beedgeable about the various service standards. All activities of thecause service selection inherently involves trust and must consider
client pertaining to the given service—including requests and recriteria that are external to any published description of a service,sponses, and communications with UDDI [UDDI, 2002] registries
whether in WSDL or DAML-S. Accordingly, this paper develops
and bindings—occur via the proxy agent for that service. In thisan ontology in which ratings of services (aggregated into reputamanner, when the client needs to bind to a service, it instantiates
tions) can be organized and shared so as to facilitate service sea proxy agent, which consults outside registries as well as repulection. This model is expressed in DAML and includes domaintation and endorsement agencies, helps find appropriate providers,
independent as well as domain-specific attributes.
records any feedback from the client, learns from the experience,
potentially shares its knowledge with the external agencies andother agents, and hopefully helps find better providers the next time
1. MOTIVATION
around. Figure 1 provides an overview of how clients typically use
Web services promise the dynamic creation of loosely coupledthese proxy agents.
information systems. However, current approaches are logically
centralized and lack key functionality, especially to locate, select,
3. REPUTATION DATA MODEL
and bind services meeting certain criteria of quality. We are developing an architecture wherein software agents serve as proxies
A distributed trust system consists of a set of principals, i.e.,
for clients and interact with one or more agencies. The agencies
the parties involved either as service provider or requester. The
help gather and disseminate service reputations and endorsements.
principals interact with each other over a set of services. A rating
However, this and other service architectures leave open some keyof a service is a vector of attribute values. The reputation of asemantic questions. Specifically, a proxy agent should be able toservice is a general opinion i.e., it aggregates the ratings of the
discover and understand new service attributes from their descripgiven service by other principals. Typically, a reputation would betions, especially as they evolve over time. At the same time, anbuilt from a history of ratings by various parties. An endorsement
agency should be able to aggregate the right information about serof a service by a principal is modeled as a boolean scalar and a time
vice quality and present it suitably formally described that they canlimit on the validity of the endorsement.
be understood by proxies.
Although we concentrate primarily on reputation, the underlying
We address these semantic questions by developing a concepconceptual model is quite similar for endorsements as well. This istual model of a service provider’s reputation for delivering quality
because an endorsement effectively states that the service being enservices. The conceptual model has a generic component (e.g., atdorsed offers high quality with respect to some selected attributes:
tribute types and common attributes such as price, on-time delivprice, reliability, and so on.
ery, and so on) and can be enhanced with domain-specific comIn our proposed approach, agents maintain and contribute ratings
ponents (e.g., closeness of itinerary to desired times, which makes
to others and discover reputations. However, the ratings are based
most sense for services in the travel domain). By combining serultimately on feedback received from their clients. Some service
vice considerations with semantic web representations, this work
qualities such as price and delay may be calculated automatically,fits into the recent activity on semantic Web services.
whereas others may require human participation. Even the latter
kind, although clearly harder to automate, can be accommodated by This paper is based on [Maximilien and Singh, 2002a].
our architecture. For such cases, the application should be designed

Clients typically have one

Client

WSAP per Web service

UDDI Registry

uses
Config 1

Service 1
Interface

WSAP Service 1
Selection is done
consulting agencies

Config 2

WSAP Service 2

Service 2Interfaceselects

Service Providers

Service 1
Implementation B

Service 2
Implementation B

Service 1
Implementation A

Service 2
Implementation A

Figure 1: WSAP operation summary
so that it is possible to receive feedback from the human user afterusage of the service. In this case, the agent will exploit this human
feedback to learn the user’s preferences.

¯ The set of endorsers of the service as well as the list of trusted
endorsers for the agent. Again, matching endorsers will bias
the reputation value.

4.

¯ The history of the service. The reputation of older services
will be affected more by previous usages.

CONCEPTUAL MODEL

A Web service represents a set of functions addressing a particular domain. For instance, a travel service might include functions
to return a list of trips for a particular airline on a specified date,time, origin and destination airport. For each service we can extract a series of generic attributes and domain-specific attributes
that apply to the service. For instance, for our travel service, ageneric attribute could be the speed at which a search produces its
results and a domain-specific attribute could be the accuracy of the
return results—e.g., whether it includes up-to-the-minute trip information. From this example and what we discussed before, it isclear that a conceptual model for the reputation of a service must
include the different categories of attributes that apply to a service.
Figure 2 summarizes our reputation model. The reputation of
the service is a function of the various attributes that matter to a
specific agent. For instance, an agent that cares more about accuracy of the trips returned would take into account this attribute
more than the relative price attribute, which may be of greater concern to another user. In essence, the relative weight given to an
attribute affects the overall reputation of the service and depends
on the user of the service. This is analogous to how the ratings ofhuman real-world services depend on the end user who is providing
the rating. A car rental service that charges more but is flexible bygiving convenient access at major airport may be of higher value toa business user than a person looking for a rental car for vacation
purposes. So within a specific domain the reputation of the service
depends on the subjective view of the user of the service on the various attributes that matters to the current agent that is proxying thisservice. Other factors that affect the reputation of a service include
the following.

¯ The relative weights given to the attributes. Each agent will
have preferences that biases it towards certain attributes andtherefore make these attributes weigh more than others for a
specific domain.
¯ The attribute aggregation algorithm. A simple weighted majority can be the normed algorithm but using different algorithms will affect the resulting reputation value.

¯ Any type of damping for the ratings as in [Zacharia and Maes,
2000] will affect the reputation value. Such damping is necessary to allow for a service’s reputation to change easily
based on its recent performance. For example, a service
that acquired a bad reputation but then become better (andstarted receiving good ratings) can have its reputation improved since older ratings matter less than the newer ones.
Figure 3 shows a UML representation of our conceptual model,
which describes the different components that make up the reputation of a service. First, a Service associates with one Reputation,
which can have many Ratings. The reputation value is determinedwith a ReputationAlgorithm that aggregates the various Attributes
that the agent determining the reputation chooses to consider. The
reputation is also affected by a History that keeps previous ratings
for the particular service. The rating for a service is determined by
the Principal in question and calculated using the RatingAlgorithm.
Any number of principals can endorse a service which might affectthe calculation of the reputation since, for instance, an endorsement
by a trusted third party can be considered of higher value than certain attribute values. As mentioned above, each attribute has a value
and range, and associates with one or more Domains. Domains act
as collections of attributes for specific types of services.

4.1 Attributes model
For each domain, the attributes in that domain are important inputs to the overall rating and therefore the reputation of a service.
Some attributes are common across domains and some are specificto domains. Each attribute has the following aspects.

¯ The value set for that attribute (and its allowed range or enumeration). For instance, an attribute such as failure rate oravailability for a service can be expressed as a percentage.
The speed of service function execution could instead be asimple bounded integer.

Collected as part ofUser Preferences

agent configuration

Reputation

A1
History

Service
Reference

Attribute
Aggregation
Algorithm

Service
Categories

Rating

DomainsDamping
FactorB1

B0

C2

Endorsement

Principal 1

Endorsement

Applicable service
domains andPrincipal 2Endorsement Principal 3

their attributes

Figure 2: Generic framework model of a service reputation*

Endorsement+timeValid1

Principal
History-privateKey-publicKey«interface»
ReputationAlgorithm

Service+wsdlInterface+wsdlImpl+uddiList

1+execute()«creates»+beginTime

1

Rating1

Conceptualcomponents
for attributesand domains*+valueReputation«interface»
RatingAlgorithm+execute()

1

1

1

*
WeightAttribute+valueDomain1

1

CrossDomain

OtherDomain

TravelDomain

1«uses»+value+range+name*RetailDomain...

MonotonicAttr

DecayingAttr+rate+function

Figure 3: UML conceptual model for service reputation...

¯ The domains that this attribute belongs to. For instance, isthis a cross-domain attribute or an attribute specific for a domain? And within each domain, some attributes will be ofgreater importance than others—this can depend on a standard definition of attributes for a domain.
¯ The weight of the given attribute relative to its domain andthe user preferences. This would determine the impact of this
attribute on the final decision regarding a provider.
¯ The characteristic of the function from attribute values to ratings. For instance, some attributes such as price are monotonic, at least in typical business scenarios. That is, the more
the price decreases the better. Generally all agents will consider price in the reputation calculation and have a preference
for lower prices. Of course, if the price were to decrease inconjunction with reductions in the values for other attributes,
the overall reputation might not improve. For instance, forthe trip service we considered, if the price attributes of trips
were to decrease while the promptness (on arrival and departure times) attribute were to become worse, then this decrease in price might not help the overall reputation of thetrip service.
The characteristics of attributes can be quite rich and need to
be further categorized. Initially, we consider monotonicallyincreasing and decreasing, S-shaped characteristics (where
there is a substantial benefit to ratings when the given attribute improves, but only if it is above a tolerance thresholdand not above a saturation threshold).
¯ The temporal characteristic of the attribute value. A possible temporal characteristic for attributes is decaying values
where the decay function can vary from exponential to a step
function. For instance, an attribute such as accuracy in thetravel domain might be acceptable to allow a range of minutes up to a certain point. That is, the trip that is scheduled
for 3:00 PM does not cause major harm if the actual departure time is 3:20 PM. However, a departure of 4:00 PM might
become unacceptable for a user who depends on an on-timearrival with a layover of 30 minutes to catch a connecting
flight. Other attributes might have values that decay moreprogressively rather than in a step-wise fashion.

4.2 Generic and domain-specific attributes
In general each domain that a service belongs to will have its own
set of attributes that apply to all services of that domain. However,certain attributes will be cross-domain attributes.
As a specific example, a service such as car rental service will
involve attributes belonging to multiple domains, such as the travel
and retail domains. Attributes such as price belong to both domains, but an attribute such as flexibility of reservation changes
has a specific meaning in the travel domain—clearly, allowing flexible changes to a travel reservation is a particular characteristic of a
travel service that might allow important service differentiation for
a particular client and the client’s WSAP.
Determining the attributes that apply to a particular domain isnontrivial, but needs be decided by the community of users andproviders as they settle upon ways to distinguish and evaluate different offerings. This typically occurs when markets form, where
different parties position their offerings according to what they believe are their key qualities: speed, price, taste, and so on. A technical challenge here is to distribute the attributes among various
domains so that an agency does not necessarily have to capture
all possible attributes and a WSAP can search for services without
consulting an excessive number of agencies.

4.3 Adding and disseminating attributes
Another important aspect that the proposed conceptual modelallows is the creation and dissemination of new attributes for specific domains. A consequence of the attributes having a common
model and the specific attributes being subclasses of the abstract
attribute model is that new attributes with unknown characteristics
can be added to the system. New attributes are disseminated via
a domain definition that could contain references to common attributes as well as to domain-specific attributes. Our conceptualmodel can be readily mapped into a common schema in a standard notational framework such as the DARPA Agent Markup Language (DAML) [Horrocks, 2002]. The existence of such a schema
would enable the description and widespread dissemination of attributes. Notice that the interesting component would be the conceptual model itself, not the notation, although agreeing on the notation is also essential.

4.4 Example
As a comprehensive example showing how the agent proxying aservice can use the conceptual model we describe, imagine a travel
reservation Web application. This application is used by agents to
set up business and personal travel arrangements. Part of this application is a facility to search, select, and reserve a car rental that will
be included as part of the overall travel arrangement. Since there
exist many car rental companies, each advertising its services on
the Web, it is easy to imagine that a common car rental Web service
interface could be created. Each car rental company would provide
an implementation of the service thus allowing its business to collaborate with others and thus integrate into coarse-grained services,such as a travel planning service. How is the WSAP proxying the
car rental service able to pick the best service for a particular client?
Using our conceptual model, the WSAP could pick a suitable
service implementation by looking at the reputation of the various implementations. The WSAP is configured with (1) the attributes that apply to the domain that the service belongs to and(2) the relative preferences that the WSAP’s principal has for various attributes. According to our model, reputation is a functionof the historical ratings provided by previous users filtered to take
into account the attributes that matter most to the current WSAP’s
principal. So, for instance, if the current user weighs price as animportant cross-domain attribute then services giving lower prices
might be selected over those giving higher prices. Of course, thereputation for the service will depend on several attributes. For instance, some attributes such as comfort and reliability of the cars
rented might matter more to a traveler who intends to use the vehicle for long subtrips than a business person on a tight budget. As
another example, though the car rental service domain attributes
overlap with a car selling service, certain attributes such as color
choice will clearly have more significance for a buyer as opposed
to a renter. Our model takes these subtleties into account, becauseof how the WSAPs are configured. Of course, the choice of which
domain a service belongs to is an important precondition that mustbe satisfied prior to agent configuration. However, we are assuming that this is done as part of classifying the services prior to them
being introduced for wide availability.
Our model is generic enough to allow the introduction of new
attributes. For instance, if the domain definition for car rental was
updated to include a new attribute such as safety, then any new
agent that was configured with this attribute could take that attributeinto account for its ratings calculations. The attribute’s values could
possibly be captured as part of a user survey or automatically bycollecting information on accidents from the car types that the car
rental company rented for certain periods and the relative safetydaml:range
daml:domain
Attributedaml:domain<<DAMLproperty>>
attrEnum<<DAMLproperty>>
attrDomain

Domaindaml:onProperty

daml:range

Restriction+ minCardinality = 1
daml:subClassOf

Generic

TravelTravel domain Price
attribute
DiscreteAttribute

DomainAttribute

ValuedAttributedaml:subClassOf
MonotonicAttributeaffordable

Restriction

cheap

daml:oneOf
TravelPriceAttrEnum

Restr iction+ minCardinality = 1

daml:onProperty<<DAMLdatatype>>
attrValue

expensive

xsd:nonNegativeInteger

daml:subClassOf

TravelPriceAttr
onProperty #domainAttr
hasValue #GenericPriceAttr
onProperty #domainAttr
hasValue #Generic

Generic domain
Price attribute

Figure 4: Part of DAML ontology for attributes showing two domain definition of price in UML [Baclawski et al., 2001]

outcome of the rentals.
A lot of engineering work goes into making a large application.
The above example is no exception. The contribution of our approach is in streamlining the attributes so that their treatment isstandardized up to a point and, where it is not standardized, placed
explicitly under the control of individual WSAPs and their principals. New attributes can added on the fly. More importantly,
the different agencies can upgrade to the new attributes or add anexisting attribute that they had previously ignored. Likewise, theconceptual model enables the WSAPs to share information directlywith each other, which extends their power further.

5.

DISCUSSION

Our architecture opens up some interesting challenges, of which
the following are germane to the topic of this paper.

Conceptual model of service attributes
Can we define a generic conceptual model for attributes reusable
across domains?
Our agents are configured to capture the wishes of the application user. The agent uses this configuration information to maximize the utility of the user. However, in order for the agent to makeintelligent decisions it will need more than just the reputation andendorsement agencies. It will need knowledge of attributes the user
cares about, such as the following:¯ The threshold of the values for attributes that the user is willing to accept.
¯ The risk tolerance of the user. For instance, the WSAP could
find a reputable service matching the user’s preferences but
because it is relatively new, selecting that service could be
regarded as higher risk than a known more mature service.
Answering the above question will enable our WSAP agents toefficiently and thoroughly capture the preferences of their WSAP
users. Further, the service selection will often need to be fast because the user may be waiting for a service to be found. Therefore, the agent must be able to make quick decisions, comparing
the user’s preferences with information provided by the agencies.

Semantics of service attributes
How can we add semantics to service attributes, thereby allowing
a WSAP to dynamically discover new attributes without having to
be reconfigured or reprogrammed?
How can the agent acquire the knowledge of new attributes thatwere not specified by its client? That is, how can the agent relate theattributes specified by its client with attributes from other agents?
The W3C’s Semantic Web initiative [W3C, 2000] is a promising
direction in capturing the semantics of service attributes.

Effects of attribute type on reputation
How should reputation be related to history of previous interactions? Should the effect of an interaction decay over time?
The notion of reputation is tightly bound to history and time.
The reputations of human services tend to vary with time and recollection. In the digital world, history and memory can be collected easily. Because of this, the notion of reputation for humans
and for agents have important differences. Some reputation systems [Zacharia and Maes, 2000] build in this decay effect. One
approach to include time in our proposed architecture would be to,for instance, associate timestamps with attribute values, thereby allowing the reputation rating to weight attributes depending on theirage. Further, since service quality will tend to change over time,
decaying the reputation helps by reducing the effect of interactions
over time, effectively increasing the currency of the evaluations.
A similar situation arises with endorsements. The goal is that itshould be as easy as how people now look into the local newspaperand select a movie by looking at the number of stars it was awarded.
Of course, a movie-goer may be biased towards a movie becauseof his knowledge of its actors, director, or producers—these intangibles will have conscious and subconscious implications to themovie-goer’s decision. This is not completely the case for the software agents. However, endorsements do affect the agent’s final decision. An endorsed service can similarly bias an agent towards aparticular service regardless of its rating. How should agents weigh
reputation ratings with respect to endorsements? What we need isa scheme by which attributes and endorsements can be systematically combined.

References
Kenneth Baclawski, Mieczyslaw K. Kokar, Paul A. Kogut, Lewis
Hart, Jeffrey Smith, William S. Holmes, Jerzy Letkowski, andMichael L. Aronson. Extending UML to support ontology engineering for the semantic web. In Martin Gogolla and Cris
Kobryn, editors, UML 2001 - The Unified Modeling Language.
Modeling Languages, Concepts, and Tools. 4th International
Conference, Toronto, Canada, October 2001, Proceedings, volume 2185 of LNCS, pages 342–360. Springer, 2001.
Ian Horrocks. DAML+OIL: a description logic for the Semantic
Web. Bulletin of the Technical Committee on Data Engineering,
25:4–9, 2002.
E. Michael Maximilien and Munindar P. Singh. Conceptual modelof Web service reputation. ACM SIGMOD Record, 31(4), December 2002a.
E. Michael Maximilien and Munindar P. Singh. Reputation andendorsement for Web services. ACM SIGEcom Exchanges, 3(1):24–31, 2002b.
UDDI. Universal Description Discovery and Integration, 2002.
http://www.uddi.org.
W3C. Semantic Web, 2000. http://www.w3.org/2001/sw/.
Giorgos Zacharia and Pattie Maes. Trust management through reputation mechanisms. Applied Artificial Intelligence, 14:881–907,
2000.