We present the WebContent platform for managing distributed repositories of XML and semantic Web data. The platform allows integrating various data processing building blocks (crawling, translation, semantic annotation, full-text search, structured XML querying, and semantic querying), presented as Web services, into a large-scale efficient platform. Calls to various services are combined inside ActiveXML [8] documents, which are XML documents including service calls. An ActiveXML optimizer is used to: (i) efficiently distribute computations among sites; (ii) perform XQuery-specific optimizations by leveraging an algebraic XQuery optimizer; and (iii) given an XML query, chose among several distributed indices the most appropriate in order to answer the query. The Web has become the platform of choice for the delivery of business applications. In particular, the popularity of Web service technologies (WSDL [17], BPEL4WS [9] etc.) and their closeness to HTML and XML, the predominant content delivery languages on the Web, has opened the way to the development of complex business applications by integrating Web services provided by different parties. This model has several advantages. From a development viewpoint, it relies on widely accepted standards, and benefits from the plethora of available application building blocks. From a business viewpoint, it allows organizing the activity in cleanly defined modules, each of which is implemented by some Web services. This enables several entities to provide implementations of a given module, and facilitates replacing one entity with another. We are currently involved in a large R&D project called WebContent [22] , whose purpose is to build and exploit large-scale repositories of rich, semantically annotated Web data. The overall setting of the project, outlined in Figure 1 , exemplifies the kinds of applications discussed above. A focused crawler (1) service returns Web documents related to specific domains, in our case, aircraft sales by Airbus and Boeing (for a continuous, online market survey), respectively, food risk information, for a consortium of food companies seeking to organize and structure information related to different food problems (contaminations, allergens etc.).
WebContent REF describes a P2P platform for managing distributed repositories of XML and semantic Web data, where various data processing building blocks are integrated as Web services.
WebContent: efficient P2P Warehousing of web data
PVLDB
PVLDB
Computer Science