Sensor Data Management

Hideyuki Kawashima
Joined from Feb. 2007～

Outline• A DBMS for Signal Streams• Future Research Direction

Ubi. Sensor N/W Apps.
MeT: Searching Physical Objects

3D-Positional Sensor

3D-Positional Sensor

Supporting Developmentof Ubi. Apps by DBMS
Ideal

Devil’s
ChoiceRealAns.１：DBMS ＋ Signal Libs.

••
DB

Impedance mismatch
Not general libs

Ans.２: All handmade
Events••Much cost to develop DBMS
Cannot reuse for others

Related WorkDataStore
Object Relational DBMS
(Informix TimeSeries)
Stream Processing Engine(TelegraphCQ, Boleralis)
KRAFTCont.
QueryFast Pattern
Insert Recog.

KRAFT Data Model:
Abstracted Data Type (ADT) for sensor
IDSensor
TypeSensor Value

1Accel.

TT

T

T

TTimeTT

T

89 12 40 20 89 31 50 89 71 22 89

2TTT

76 45 12 30 77 33 76 82 29 62 76 31
23 90 78 23 11 59 98 24 49
21 10 15 21

2Temp.

3Humid.

INTEGER

TEXT

T

14855 23

29 11 48 79 21 43

SENSOR (Multi-Dimensional Time Series)

Features of KRAFT• Fast Data Insertion• Time Series Analysis• Periodic Query Execution

Magic of Acceleration-- Data Insertion Mechanism -WAL is the bottleneck
RAMwith UPS: PersistentDev.
RAMRAM
InterleavingMovement
Rand. AccessAsynch. PageDiskNaïve MethodDiskWALRAMDiskKRAFT

Data Insertion Speed
Mili Sec
4000

100 ins. for each host
（RPM of disk = 7200）

3500
3000
2500

PostgreSQL

2000
1500
1000

KRAFT

500
01234567891020# of Concurrent Client Accesses

50100

Features of KRAFT• Fast Data Insertion• Time Series Analysis• Periodic Query Execution

Data Model (again)• Object Relational（ADT for sensor）
ID

Name (text)

0KawashimaSensor

Select * from TABLE where id = 1;
Arrival time
Data ValueInsert into TABLE values …
Append into TABLE values …
Delete from TABLE where id = X

Funcs for SENSOR ADT• Aggregation (as usual DBMS)– MAX, MIN, CNT, AVG, SUM,

• Signal Processing– FFT, Wavelet• Similar Sequence Retrieval– DTW, Euclid• Sliding Window

Features of KRAFT• Fast Data Insertion• Time Series Analysis• Periodic Query Execution

Periodic Query Execution(Continual Query)• After query
registration, answers
are periodically
returned• Human passive,
DBMS active (usually,human active, DBMSpassive)• A PQ is realized as athreadProblem on
Periodic Data Acquisition
Problem： Delay of query activation
Time
Ideal Start Time (PST)
Real Start Time (RST)Approach• Cooperation of CQ and OS scheduler
4000

DBMS

OS(Thread Scheduler)

Delay (Micro sec)

3500
3000
2500

Round Robin

2000
1500
1000

KRAFT

500
0100

200

300

400

500

600

Concurrency

700

800

900

1000

Outline• A DBMS for Signal Streams• Future Research Direction

Research Direction (1/2)• Storage for StreamSpinner (StreamProcessing Engine, explained later)– From toy apps. to real sensor apps.
Data StreamStreamSpinner
Real-Time Archiving

Environmental data stream(Meteorological data)

KRAFT

Research Direction (2/2)• Data Mining Meteorological Data
– Classification of Weather Map by PressureWeatherMap
Database

Except for
Winter Type
Winter Type (西高東低）

Summary• KRAFT: a DBMS kernel for signal streams– Fast data insertion– Signal processing functions– Periodic query execution– http://sourceforge.jp/projects/kraft• Future Direction– Integrating StreamSpinner– Data Mining for Scientific Data

Funding and Outputs•

Funding– MEXT Grant-in-Aid for Young Scientists (B)– IPA Exploratory Software Project•

Publications– Hideyuki Kawashima，”KRAFT: A Real-Time Active DBMS for SignalStreams”, Proc. of 4th International Conference on Networked Sensing
Systems (INSS'07) pp. 163-166，June 2007
– Hideyuki Kawashima，Yutaka Hirota，Michita Imai，”MeT: A Real World
Oriented Metadata Management System for Semantic Sensor
Networks”，Proc. of 3rd International Workshop on Data Managementfor Sensor Networks (DMSN'06)，pp．13-18, September 2006．
– Hideyuki Kawashima，Michita Imai，Yuiciro Anzai，”Providing
Persistence for Sensor Data Stream by Remote WAL”，Proc. of 8thInternational Conference on Data Warehousing and Knowledge
Discovery (DaWaK'06)，Lecture Notes in Computer Science, Vol．4081，pp．524-533，September 2006．