NTCIR13-Lifelog
Task Introduction
12314

Cathal Gurrin , Hideo Joho , Frank Hopfgartner , Liting Zhou , Rami Albatal
1234

Dublin City University, University of Tsukuba, University of Glasgow, HeyStaks

Life Experience
Digital Self

How to create a digital lifelog of life
experience without manual input…
activities, experiences, interactions,emotions? … and make it useful…

Narrative Clip camera : $200

Quantified
Self… self
knowledge
thoughnumbers

Motivation for NTCIR-13Task••

NTCIR-12 Lifelog pilot task attracted 8 participants across two sub-tasks•

LSAT - Semantic Access (known item search)•

LIT - Insights Task

NTCIR-13 Lifelog-2 is now a core-task with some significant changes:•New rich data (90 days, 2 people), less anonymisation•

Two new sub-tasks:•

LAT - Lifelog Annotation Task•

LEST - Lifelog Event Segmentation Task•

Staged tasks (LAT in phase 1, and 3 others in phase 2)•

Hope to also have a co-located workshop in Europe

Data Summary•

Three months of rich lifelog data from two people•

2 x 45 days each, all day data (about 20GB)•With accompanying event segmentation, dietannotation and automatic visual-annotation•Data released via individual and organisational
agreement (as at NTCIR-12)•

Minimal anonymisation of the data

01

Wearable Multimedia

Human Biometrics

1,500 images per day fromthe Narrative wearable
camera. Accompanying
concept annotations.
Periodic audio. Manualphotos captured. Music
listening history.

24x7 heart rate, galvanic
skin response, calorie burn,steps, skin temperature.
Daily blood glucose level &blood pressure. Weekly
cholesterol and uric acidmeasurements.

MINUTE AS
THE UNIT OFRETRIEVAL

02
04

03
Information Access

Human Activity

Onscreen reading,
keystrokes on keyboard,mouse movements,
computer activity, webpages viewed.

Physical activities (walking,running, transport, etc..),locations visited, food eaten,mood.

Blood Pressure,
glucose, uric acid,cholesterol.
24x7 heart rate &GSR, calorie burn,steps.

Wearable camera
data from
Narrative Clip 2(1,500 per day).
Manual Photos
Periodic Audio
Music listening
records

Words written andread, grouped intominute-long
documents withTFs.
Mousemovements.
Web pages
browsed.
Computer activity.

Physical activities
Locations
Food eatenMoodSub-tasks•

EARLY TASKS••LAT - Lifelog Annotation Task

LATE TASKS

DonatedAnnotations•

LSAT - Known-item search (repeat from NTCIR-12)•

LEST - Lifelog Event Segmentation•

LIT - Lifelog Insights Task

LAT TASKLSAT Task

LIT Task

LEST Task

LAT - Lifelog
Annotation Task. A
multimedia data
analytics task toautomaticallyannotate the visual
lifelog data.

A known-item
search task.
Supports bothautomatic andinteractive retrievalsystems.

Insights task. This
year the focus ison making themed
diaries of lifeexperience. Not
evaluated bymetrics.

Eventsegmentation task
for lifelog data,
evaluated against
a groundtruth
generated by thelifeloggers.

Feb’17

Jul’17

Jul’17

Jul’17

Phase 1 Data

NTCIR-13

Phase 1 raw data
releaseWelcome to Tokyo or
co-located event

Other Runs Due
LSAT, LEST due. LIT
abstract only due

06
NOW01

Nov
1605

02
04

Feb
1703

Dec
17

Aug
17

Jul17

Apr17

Results
LSAT, LEST run
results distributed

LAT Runs Due Phase 2 & API
LAT Annotation Runs
are due on 15 Feb’17

Rich data usingoutputs of LAT &baseline search API

01

Data Gathering

03

P2 dataset enrichment

05

Evaluation Phase

02

P1 experiments

04

P2 experiments

06

Paper writing

LAT - Lifelog Annotation
Task
A multimodal annotation task, aimed at multimedia data analysts toannotate automatically annotate the visual lifelog data with conceptlabels, environment labels and activity labels.

Kahneman et al. A survey method for characterizing daily life experience: The day reconstruction
method. Science, 306(5702):1776–1780, 2004.

LSAT - Known-item Search
A known-item search task in which participants have toretrieve a number of specific moments in a lifelogger's life. We
define moments as semantic events, or activities that
happened throughout the day. LSAT can be undertaken in aninteractive or automatic manner. We will provide a base-line
retrieval engine (via API) for participants who want to build
an interactive LSAT system.
Example search tasks include:
Find the moment(s) where I was boarding an A380.
Find the moment(s) where I am in my kitchen.
Find the moment(s) where I am playing with my phone.
Find the moment(s) where I am preparing breakfast.

Event Segmentation

LIT - Lifelog Insights Task
The aim of this subtask is to gain
insights into the lifelogger's life.
Participants are requested to generate
new types of visualisations and insights
about the life of the lifeloggers bygenerating a themed diary. This task is
not evaluated in the traditional sense,
but participants will be expected topresent their work in a special session
at NTCIR. An event segmentation will
be defined for the data.

from:NTCIR-13 Lifelog Organisation TeamWebsite: ntcir-lifelog.computing.dcu.ie
Contact Email: cathal@gmail.com