Meme Media and Knowledge
Federation
Yuzuru Tanaka, Jun Fujima,and Micke Kuwahara
Meme Media Laboratory, Hokkaido University
N13 W8, Sapporo 0608628, Japan{tanaka, fujima, mkwahara}@meme.hokudai.ac.jp

The Web as a repository of a hugevariety of knowledge resources• The Web is becoming a huge open repository ofknowledge resources that cover almost all areas– knowledge resources• What can be dealt with by computers, including data,
documents, tools, and services, for the sharing and reuse byusers.

• What prevents our maximum utilization of Webresources?
– the difficulty for a user to find what he or she isinterested in– the difficulty for a user to make mutually related Web
resources work together

R&D efforts• the difficulty for a user to what he or she isinterested in– Big projects in the US, Europe, and Japan, such as
Digital Library Initiative (DLI) Projects in US, andInformation Grand Voyage Project in Japan• the difficulty for a user to make mutually related
Web resources work together– no major projects except those on GRID computing,• which basically focuses on the interoperability of distributedresources to perform routine or frequently requested jobs.

Knowledge federation as a key
technology to develop• “Federation” denotes ...
– the process of combining multiple technology elements into asingle virtual entity• 2 origins in IT– a federated database architecture (Dennis Heimbigner,1985):• A collection of independent database systems are united into aloosely coupled federation in order to share and exchange
information.

– federation of services (Bill Joy, late 90s) :• A federation is a set of services that can work together to perform atask• Knowledge federation (Yuzuru Tanaka, 2004)– flexible, ad hoc, and instantaneous selection, customization, andcombination of knowledge resources by users– The 1st International Conference on Knowledge Federation, Oct.
20-22, 2008, Duvrovnik, Croatia

Extending the Web
Web 2.0
Collaborative and/or
semantic
reorganization ofWeb resources tostructurally enrichrelations among
Web resources

Knowledge federation

Collaborative reediting of Web resources forcomposing new ones to structurally andfunctionally enrich Web resources

Federation is different fromintegration• Federation– assumes an open networked environment of diverse,
heterogeneous, autonomous, and distributed
resources, and– deals with open scenarios of information processing.

• Integration– targets local and centralized management, andinteroperation of resources in a closed environment,and– deals with closed scenarios of information processing.

Why ad hoc knowledge federation?

Think

Whenever wehave a new idea

and use this to spark
further ideas

Try
we try it out,

Seeobserve andevaluate the result,Acceleration of Creative Thinking

Think
To try an idea, weneed to represent it
as an executable KR(knowledgeresource), using
available KRs.

TrySeeTo evaluate thetrial results, weneed to apply
various evaluation
and analysis tools.

Ad hoc knowledge federation andthe reuse of federation results• Immediate trial of a new idea requires– rapid composition of a new tool from available
resources,– its application to a data set extracted fromavailable resources.
– repetition of such processes.

• Further reuse by other people requires– republication of these composed tools andextracted data sets as web resources

A grand challenge of knowledge
federation technologies• Could one compose such a complex application as adrug design system just through ad hoc federation ofavailable Web resources?
Components existas Web resources.
Ligand seq.

Liganddesignvisualization
visualization
Receptor
input
Input both a target
sequence and a similar
template structure
Visualization ofeach candidate

ligand input

But yes ina near future.

conflict
detection

Structure
prediction

Target seq.

Not possible now,

DockinganalysisOthersolvers

electrostatic energy
analysisScience Infrastructure•

new research methodology: ‘data-based science’, or ‘data-centricscience’– systematic data acquisition first,– then data analysis for hypothesis validation• genomics, proteomics, brain science, clinical trials, nuclear physics,
astrophysics, material science, meteorology, and seismology•Leading to– a large number of huge, independent accumulations of data, and– a great variety of data analysis tools.

•Requiring knowledge federation operations– extraction of appropriate data sets from multiple seemingly unrelated
sources,– their compilation or customization, and– the application of data analysis and/or data visualization tools to thesedata sets.

•Such operations are inherently performed in an ad hoc manner, andshould be executable rapidly enough to follow up on fleeting ideas.

Science infrastructure vs.
conventional e-science• Science infrastructure– requires a new generic technology for ‘knowledge
federation’• as well as the development and extension of databases,
simulators, and analysis tools in each area.
• to meet the dynamically changing demands of creativeactivities.

• Conventional e-science– For routine or frequently requested jobs
• work-flow and/or resource-orchestration technologies• based on GRID computing.

Taxonomy:
Knowledge Media, Meme Media,and Knowledge Federation

Knowledge resources, andknowledge media• Knowledge resources– What can be dealt with by computers, including data,documents, tools, and services, for the sharing andreuse by users• Knowledge media– externalize knowledge resources as compound
documents, i.e., documents with• embedded nonfunctional multimedia contents and/or• functional contents such as application tools and services– The Web is an example of knowledge media.

The Web as knowledge media

serverPublication
infrastructure
servercompound documentwith embedded tools
and services

Browse + download/uploadclientMeme mediameme pool

userB

compounddocumentdrag & drop
userA

drag & drop

redistribution
With interoperation among
functional components

reeditinguserCKnowledge federation to make theWeb work as meme media• Knowledge federation introduces memetic evolution tothe Web• Biological evolution is based on mechanisms– Replication / Recombination / Mutation / Natural selection• Same is true with the evolution of knowledge resources(Web resources)– Replication / Reediting / Evaluation by a society or a community– Reediting• Easy for multimedia documents• Difficult for compound documents with functional components– Knowledge federation accelerates collaborative reediting of Web
resources• Web resource as a meme

Meme Media as Basic
Enabling Technologies

Meme Media Architectures:IntelligentPad and IntelligentBox• 1987-1992:– Component-based media
architecture IntelligentPad (1989)• pads as media components• 1993-2001:– IntelligentPad as a Meme media
architecture for users to reedit
and to redistribute intellectual
resources (1993)– 3D meme media architectureIntelligentBox (1995-)• boxes as media components

IntelligentPad as Meme Media
Pads are pasted together todefine a Compound Document.
P1P2

Each pad wraps an object with astandard display representation
and a standard logical interface.

P3
P2

P3

P1
Each pad has a set of connection jacks
called slots and a single pin plug to beinserted to one of its parent’s slots

How to apply meme media toWeb resources?

Wrapping web resources asmeme media objects
The WebWebapplications

Web services

(1)

(2)

(4)

(3)

LegacyApplications

wrapping

Meme Media Objects(5)

Composition(6)

Meme Media and a Meme Poolt

How to wrap different types ofknowledge resources?
The WebWebapplications

Web services

(1)

(2)

(4)

(3)

LegacyApplications

wrapping

Meme Media Objects(5)

Composition(6)

Meme Media and a Meme Poolt

Wrapping and Linking of Web Applications
CHIP (2003) for Dynamic Linkage among Web Applications

Internet
2. Functional
Linkagethrough

Wrapper
NOT
INTEROPERABLE

WebApplications

1. Wrapping

Drag&Dropthrough
Mouse Operations

Wrapper

Integrated ToolIntelligentPad EnvironmentLinkage between Web clips fromdifferent navigations (C3W, 2004)
C←B

F←exractTxtValue(B)C←setTxtValue(./@value, F)

How to wrap different types ofknowledge resources?
The WebWeb
applications

Web services

(1)

(2)

(4)

(3)

LegacyApplications

wrapping

Meme Media Objects(5)

Composition(6)

Meme Media and a Meme Poolt

Proxy pads andserver-side meme media• Server-side Meme Media（2005） enabled users– to automatically generate proxy pads for accessing
arbitrary Web services,– to paste them together for their composition, and– to convert the composite pad to a new Web service.
i-applidownload
SOAP

DeviceLocation
ServiceIP
SOAP
Server(IPSS)
Pad
applications
IntelligentPadsystem

WebCameraServiceHow to wrap different types ofknowledge resources?
The WebWebapplications

Web services(1)

flattening

(2)

(4)

(3)

LegacyApplications

wrapping

Meme Media Objects(5)

Composition(6)

Meme Media and a Meme Poolt

How to wrap different types ofknowledge resources?
The WebWebapplications

Web services

(1)

(2)

(4)

(3)

LegacyApplications

wrapping

Meme Media Objects(5)

Composition(6)

meme pool

Meme Media and a Meme Poolt

Piazza as a meme pool system(≠ the Web)bookmarks

Piazzapiazzaeditorchatwindow

Upload anddownload ofpads by dragand-drop (1998)

Wiki Piazza (2004)• Wiki piazza system works
both– for resource providers toregister new computing
resources, and– for resource consumers
or users to look up andto use availableresources.

Repository service
to register computing resources wrapped as pads
Lookup service
also composed by Wrapping of Wiki Search and Google
Search: Similarity searchPiazza based on Wiki
Wrapping Wiki to work as a Pad Repository(1)click(2)clip(4)clip(3)click

ClipboardPad#output#inputAfter defining slots to the ClipboardPad, we use only this pad.

How to make the Web work as ameme pool?
• The 2008 version of IntelligentPad: a web-top system– fully exploits Microsoft Silverlight technology, and runs on
Internet Explorer 6 and 7, Mozilla Firefox, or Safari browser
empowered by Silverlight plug-in.
– no need to install any IntelligentPad kernel on clientsNew features of the new version• no IntelligentPad kernel running on clients– Users do not recognize any difference between web
resources and composite pads.
– Composite pads can be also registered in HTTP
servers as extended web resources.

• canvas-free pads
– Any graphical object including a line segment can betreated as a pad.
• SVG (Scalable Vector Graphics) to describe graphicalobjects.
• A child pad on a canvas-free pad is bound by the local
coordinate system spanned by the latter pad,The Memetic Web
legacyapplications
Web
services

The Web server-

proxypads

wrappingside IPWeb
applications C3W

portalcomposite
pads andboxes
padcomposition

immediate ad hoc compositionand orchestration

The Memetic Web

achieved
plannedThe Memetic Web
proxy pads
Web
service A

reediting &redistributionWeb
service B

The Webclipping

RIA (Silverlight) tech.

copypublish
reuse in local
environments

The Memetic Web

Only requires Silverlight-empowered browser!
browsing, using, reediting and redistributing envt.

EU's FP6 Integrated Project
ACGT (Advancing Clinico-Genomic Trials on Cancer)•Trial Outline Builder: a subsystem of ObTiMa– Joint development with the Department for Pediatric Hematology andOncology at the University Hospital of the Saarland, and Fraunhofer IAIS
Architecture overview
Trial Outline Builder
EventsACGT Trial
Builder(2008- )ICEIII

50 Gy

IIII

II

50 Gy

IIII50 Gy

III

II50 GyICECEICEII50 Gy

0Canvas-free IntelligentPad
We b se rvice AP I
HTTP re que st

P a tie nt datama na ge me nts ys te m

EventsScratchboard

Scratchboard

Othe rre pos itory

TrialsScratchboard
1,0

relapse free survival,9

Trial 1
Trial 2II

I

50 Gy

drag & drop

IIICE

IIII

I
II,6I

I

I

50 Gy

IIII

II

50 Gy,5,4ICE,2
Trial 4

Life table,10,0CE

Trial 5I

III

I

50 Gy

IIIII

II

50 Gy

012345

Trial 60

Trial 80

Trial
Trial8 978910CoxRegression…

IIII

50 Gy

III

II50 Gy

Visualisation
Board

Trial 96time [years]

Trial 750 Gy

Histogram,3CE
copyResults,7Trial 350 Gy

CEICEQueryboard,8Web-top Trial Outline
Builder (2008)

Meme Media and Knowledge
Federationvs.
SOAs and Web Mashups

Related Technologies (1)Service-Oriented Architecture (SOA)• Two similar technology trends:– Science / Business Cloud-based SOA• Provides processing power, services, middleware, developers’
toolkits through the Web– Science / Business GRID-based SOA• Provides HPC power, services, and service-composition workflowdefinition / execution / management systems.

• Both emphasize:– definition, execution, and management of routine or frequently
requested jobs
– not user-centric, immediate, unforeseen ad hoc composition• Therefore both provide:– tools for editing internal representations of service compositions,– not tools for directly reediting compound documents to compose
new knowledge resources

Related Technologies (2)Mashup Technologies• Mashup: user-driven micro-integration of Web data
– i.e., data merging / data feeding / data joining / data filtering /data annotating– Traditional mashup tools are based on workflow composition ofWeb services to construct a new Web application or service• Strengths– They are starting to address RIA (Rich Internet Applications).
– Some tools (e.g., Mash Maker and Lotus Mashup) allow users toclip widgets directly from Web applications.

• Weaknesses– Widget extraction and composition use different environments.
– Widgets cannot be combined with local tools to compose newapplications (other than limited support for de facto standard
office tools).

Knowledge Federation beyond
the WebService Selection and Federation
Willcom W-SIM module(mobile phone module + antena +phone directory)(W25.6mm×D42.0mm×H4.0mm )Web

Web-based access
Mobiles
Coordinator

e.g.ZigBee
Networkproximity-based
accessIntelligent sensor modules(1.5cm)3
16bit PC, solar battery, sensors
50m WiFi range
10 yrs operation w/o any
maintenanceWeb-based FederationWeb

Web-based service
selection & federation
Web-based access
Mobiles
Coordinator

e.g.ZigBee
Networkproximity-based
access

Proximity-based service
selection & federation

Knowledge Federation between aSensor Network and a Robot
Web application showing sensor
data of a specified nodeclippingcoordinator

ZigBee Protocol

sensor nodes

composition

sensor node

8bit PCsensors(light, temperature,3D acceleration)
100m WiFi range

Coordinator(USB)

robot withconversation
capability

Concluding Remarks (1)• The current Web– a huge repository of knowledge resources ofalmost all areas.

• The problem is how to make it easy
– to select some set of resources including web
resources and local ones,– to customize them, and– to make them work together.

Concluding Remarks (2)• Knowledge federation as a key technology for people– to freely and immediately reedit some of the Web resources,– to compose a new knowledge resource for his or her own reuse,
and– to publish it again as Web resources..

• The 2008 version of IntelligentPad extends the Web tothe memetic Web.
– Users can• freely publish both web resources and composite pads into thememetic Web,• extract resource fragments from some resources in the memeticWeb,• make them work together to compose a composite pad, and• publish it into the memetic Web for its further reuse by others.

For further information on meme media
architectures and applications
Tanaka, Y.
Meme Media and Meme
Market Architectures:
Knowledge Media forEditing, Distributing, andManaging IntellectualResources.
IEEE Press & WileyInterscience, NJ (2003)