# Presentation: 134423

## Intelligent Aggregation of Information  or Challenge of Semantic Heterogeneity orWhere did NASA’s Mars Climate Orbiter go?Stuart Madnick   {smadnick@mit.edu}Context Interchange Systems Laboratory (CISL)MIT Sloan School of Management

- © S. Madnick, 2002

## MIT eBusiness & Financial Services

- Strategy

- Aggregators

- Universal Financial Aggregation  ( UFA )

- Application

- eCommerce & Financial Services

- Technology

- Web wrapper & Context mediator

## Overview

-   _Aggregator definition, types & examples_
-   Enabling technologies
-   New emerging aggregators
-   Summary

## Aggregator Definition

- Transparently collects information
-   with _or without_  prior source arrangement
- 	resolves contextual differences
-   Performs analysis
-   	not merely gather
-   Represents user
-   acts as agent for user

## Aggregator Types & Examples

- Comparison Aggregator
- aggregates book price information
- Relationship Aggregator
- aggregates frequent flier milage programs
- Intra-organizational Aggregator
- aggregates separate departmental calendars
- Inter-organization Aggregator
- aggregates information about a company from multiple sources (financials, news, etc)

## www.amazon.com

## www.books.com *

- * Acquired by Barnes & Nobles

## www.mysimon.com

## www.evenbetter.com *

- * Now called DealTime.com

## Mega - Aggregators

-   Note:  multiple book price aggregators
-   using differing sources
-   giving differing results !
-   What about an aggregator of aggregators
-    A   Mega- Aggregator   ... more later ...
-   Aggregators not just about prices ...

## www.usair.com

## www.maxmiles.com

## What is MaxMiles

-  You give it all your frequent flyer info, it . . .
-   Does mileage management for  you . . .
-   Gathers all your frequent flyer info
-   Sends a personalized report via email
-   Updates reports on web site daily
-   Searches for missing mileage credits
-    Analyzes most current milage offers

## From US Airways site

- "Access to this information is subject to the rules in the Dividend Miles Membership Guide. In addition, by using this site to access your Dividend Miles account, you agree that you will use this site in a manner consistent with the Dividend Miles Membership Guide and you further agree _not to allow access to this site__ __to any  third party_ by revealing your access code to any third party for any reason. Failure to comply with the foregoing restrictions on the use of this site shall be grounds for the termination of your access to this site and/or your membership in the Dividend Miles program."

## “Power of Attorney” on MaxMiles Site

- “Power of Attorney” on MaxMiles Site

## Aggregators are your agents

-   Note:  MaxMiles is merely aggregating
- 	your information for you
-   More examples of relationship 				aggregators later
-   Many more types of aggregators  . . .

## Intra- and Inter-Organizational Aggregator

- Integrated Spreadsheet

- Sources

- Internal

- External

## Aggregator Types & Examples

- Comparison Aggregator
- aggregates book price information
- Relationship Aggregator
- aggregates frequent flier milage programs
- Intra-organizational Aggregator
- aggregates separate departmental calendars
- Inter-organization Aggregator
- aggregates information about a company from multiple sources (financials, news, etc)

## Aggregatees

- Books.com,  MySimon.com,  EvenBetter.com
- Barnes and Noble,  Borders,  Amazon, . . .
- MaxMiles
- United,  American,  TWA,  US Air, . . .
- Universal Financial Aggregator (UFA)
- your banks,  your brokers,  your billers , . . .
-  Reality:  Aggregat_ee_s -> Aggregat_or_s

## Overview

-    Aggregator definition, types & examples
-   _Enabling technologies_
-   New emerging aggregators
-   Summary

## MIT COntext INterchange (COIN) Project

- Appli- cations

- OUTPUT PROCESSING
- ODBC Driver
- Web -      Publishing

- CONTEXT MEDIATION
- *  Automatic conflict detection and conversion
- - Derived data
- - Source selection
- - Source attribution

- TRUSTED AGENTS

- INPUT PROCESSING
- * Automatic web wrapping
- - Semi-structured text
- -Multi-source query plan and execution

- Browsers

- Data bases

- APPLICATIONS:  Financial services, electronic commerce, asset visibility, in-transit visibility.

- Sources

- Web Pages

- Receivers

## Emerging Technologies

-   Web wrappers
-   _Extract_ selected information from web
-     ( both HTML and XML )
-   Allows web to be treated as large database
-   Context mediators
-   _Resolve_ semantic (meaning) _differences_
-   Enable meaningful aggregation    & comparison

## E.g., Semi-structured Web data: Intel SEC filing

- Net Income

## Web Navigation

- Home page

- Target page

- SEC Site

- Filing with Intel Net Income

## Web-Wrapper Technology

- Web page  spec file *

- Select Edgar.Net_income
- From Edgar
- Where Edgar.Ticker=intc
- and Edgar.Form=10-Q

- Ticker

- Net Income

- INTC

- 1,983

- User or Program
- (SQL Query)

- Web Wrapper Generator

- Data record returned

- * Spec file contains:
-   Schema definition
-   Navigation rules
-   Extraction rules

- Web site ( HTML or XML )

- SQL
- side

- HTTP
- side

## Role Of Context

**  **CONTEXT VARIATIONS:
- 	-  GEOGRAPHIC  ( US vs. UK )
- 	-  FUNCTIONAL   (CASH MGMT vs. LOANS )
- 	-  ORGANIZATIONAL  ( CITIBANK vs. CHASE )

- Data:
-   Databases
-   Web data
-   E-mail

- ?

- $

- 02-01-03

- 01-02-03

- 03-02-01

## Example : Context Differences

- ( from multiple web sources)
- Daimler Benz  ( DAI )  Key  Ratios

- 			_P/E Ratio_	_EPS_		_Dividend_
- ABC			11.6				0.29
- Bloomberg	5.57		15.32		8.127
- DBC			19.19		4.36		0.899
- MarketGuide	7.46		10.83		0.47

## The 1999 Overture

- Unit-of-measure mixup tied to
- loss of $125Million Mars Orbiter
- 	“NASA’s Mars Climate Orbiter was lost because engineers did not make a simple conversion from English units to metric, an embarrassing lapse that sent the $125 million craft off course.  . . .
- 	. . . The navigators ( JPL ) _assumed metric units_ of force per second, or newtons.  In fact, the numbers _were in pounds_ of force per second as supplied by Lockheed Martin ( the contractor ).”
- Source: Kathy Sawyer, *Boston Globe*, October 1, 1999, page 1.

##  Context Interchange Technology

** **Context Interchange Technology

- Context
- Mediator

- Concept:  Length
-    Meters       Feet

-           function( )
- meters                   feet

## Example:  Multiple Sources of Financial Data

**Example:  Multiple Sources of Financial Data**

- Company Name

- Company Name

- Net Income

- Net Income

- Sales

- Sales

- DAIMLER-BENZ AG

**346,577**

**56,268,168**

**615,000,000**

**97,737,000,000**

- O&A DEM-USD Exchange Rate

- 1.00 German Mark= 0.58 US Dollar as 12/31/93

- WorldScope

- Disclosure

- OANDA
- Web Server

- Context Mediation
-        Services

- Users  & Application Systems

- Net Income

- Company Name

- Sales

- DAIMLER-BENZ

**614,995**

**97,736,992**

- Datastream

- Wrapper Services

- *

- *

- *

- *

- *

- DAIMLER BENZ CORP

## Some Context Differences

|  | Disclosure | Worldscope | DataStream |
| --- | --- | --- | --- |
| Currency Used | Country of Incorporation | USD | Country of Incorporation |
| Currency Symbols | 3 Letters | 3 Letters | 2 Letters |
| Scale Factor | 1 | 1000 | 1000 |
| Company Names | Disclosure Names | Worldscope Names | DataStream Names |
| Date Style | American with ‘/’ as separator | American with ‘/’ as separator | European with ‘-’ as separator |

## Domain Model  ( partial )

- Inheritance
- Attribute
- Modifier

- Some currency context possibilities:
-   Currency is stated explicitly as part of record
-   Current not stated, but the same for all (e.g., US $)
-   Currency not stated or constant, but inferred by country

## System Demonstration

**Q6.  Scenario: **Using Context Interchange, the financial analyst can look at the Disclosure data using Datastream Context.
**Query: **Find out from Disclosure what Net Income for DAIMLER-BENZ was.  Use Datastream Context.
**Capabilities Demonstrated: **
- 	Ability to perform Scale Factor Conversion, Date Format Conversion, Company Name Conversion.

**Single Source Queries with Mediation**

## Demonstration – context2.mit.edu

## Conflict Detection and Mediation

**Date convert**

_**Mediated Query in Datalog**_

**Scale factor convert**
**Name convert**

## Mediated SQL Query & Result

**Adjust scale factor**

**Date format**** **
**conversion**

**Name conversion**

_**Final results**_** **_**–**_** **_**from**_** **_**Disclosure but in Datastream context**_

_**Mediated SQL Query**_

## Overview

-   Aggregator definition, types & examples
-   Enabling technologies
-   _New emerging aggregators_
-   Summary

## Concept of a Universal Financial Aggregator ( UFA )

- User

- UFA

- Citigroup
- Credit Cards

- FidelityBrokerage

- FleetBanking

- BancOneCredit Cards

## Universal Financial Aggregator (UFA) Demonstration

## Aggregated Report produced ( top part )

## Aggregated Report produced ( bottom part )

## Current UFA Offerings (examples)

- Account Aggregation Technology Suppliers
-   Yodlee
-   VerticalOne
-   eBalance
- Account Aggregation Offerings
-   Chase, Fleet, Citibank (myCiti.com), . . .
-   Merrill-Lynch, Fidelity Investments, . . .
-   Yahoo, AOL, . . .

## “Aggregation  . . . The ‘killer app’ in the 

- “Aggregation  . . . The ‘killer app’ in the
- e-finance revolution”

- http://www.tfconferences.com/conferences/AAG01/index.html

## “After Aggregation” Services

- FI #2

- FI #3

- FI #1

- User

- UFA

- First Call

- Excel

-  Advisors
-  Wizards
-  Agents

- Account Analyst

- Asset
- Allocation

- Financial  Sources

- Other Internet Sources

- Intranet Sources
- and applications

- “After Aggregation” Services

- User
- Information

-   _Emergence of Open Aggregation_

## Advisors:  Stock holdings re-aggregate

## Wizards:  Money Market (MMA) example

## MMA is also an aggregator, of MM rate sources

## Multi- Mega- Aggregators !

-  The Universal Financial Aggregator (UFA) :
-   _Aggregates_ your financial information
-   several ways: totals and stock holdings
-   Also uses the Money Market _Aggregator_
- 	which draws on multiple MM sources
- 	including sources that are, in fact,
- 	 _aggregators_ of some MM sources

## There may be value in aggregation but...The REAL VALUE is after aggregation services

- Books.com - How to price my books?
- MaxMiles.com - Who flies where and when?
- Money Market/CD rates - Will my offering be effective? Are the holes in the market?
- UFA - Can I manage my finances better?

## Overview

-   Aggregator definition, types & examples
-   Enabling technologies
-   New emerging aggregators
-   _Summary_

## Key Insights and Implications

_**Insights**_
-   Everyone impacted
-   Aggs not limited to price comparison
-   Aggs can be combined and aggregated
-   Aggs create new info spaces
-   After-agg even more important
-   Business models and market conditions continue to change
-   New technologies becoming available

_**Implications**_
-   Be part of strategy planning
-   Identify new Agg applications and business opportunities
-   Entirely new types of applications can be defined
-  Who will own this space
-   This must be exploited
-   Must be prepared for changes
-    Start exploring them now

## The 1805 Overture

- 	In 1805, the Austrian and Russian Emperors agreed to join forces against Napoleon. The Russians said their forces would be in the field in Bavaria by _Oct. 20_.
- 	The Austrian staff planned based on that date in the _Gregorian calendar_.  Russia, however, used the ancient _Julian calendar_, which lagged 10 days behind.
- 	The difference allowed Napoleon to surround Austrian General Mack's army at Ulm on Oct. 21, well before the Russian forces arrived.
- Source: David Chandler, *The Campaigns of Napoleon*, New York: MacMillan 1966, pg. 390.

## Current Projects ( partial list )

-   Aggregation strategies (e.g., Open aggregation)
-   Regional & Global Aggregation:  Cultural & legal
-   Multi-enterprise eBusiness: Trust & relationships
-   Financial services/Internet support for SME’s
-   Future Financial Services (e.g., Wireless aggregator)
-   Context Mediation & Web data extraction *
-   Data Quality & What XML will and will _not_ do
-   Internal data standards efforts (e.g., EDSI)
-   Other new technologies (e.g., Mobile aggregation)
-        						* Patents awarded in 1999

## For Further Information and Contacts

- MIT Research:
-  http://context2.mit.edu and http://context2.mit.edu/aggregation
-  http://web.mit.edu/tdqm
- MIT  Technology Licensed Software:
-   http://web.mit.edu/smadnick/www/patents.html
- Email Contact:
-   Stuart Madnick at smadnick@mit.edu
-   Michael Siegel at msiegel@mit.edu

## Acknowledgements

-   Research funded, in part, by
-   Banco Santander (BSCH)
-   Citibank
-   DoD Advanced Projects Agency (DARPA)
-   Fleet Bank
-   Merrill Lynch
-   MITRE Corporation
-   PricewaterhouseCoopers
-   Singapore-MIT Alliance (SMA)
-   Suruga Bank