NTCIR16-Lifelog4
Kickoff
Cathal Gurrin (Dublin City University, Ireland)Frank Hopfgartner (University of Sheffield, United Kingdom)
Duc Tien Dang Nguyen (University of Bergen, Norway)
Hideo Joho (University of Tsukuba, Japan)
Thanh Binh Nguyen (HCM University of Science, Vietnam)
Graham Healy (Dublin City University, Ireland)
Liting Zhou (Dublin City University, Ireland)1Increasing Relevance in 2021

16
ImageCLEF - Lifelog
Annotation & Summarisation
Challenges

ImageCLEF - Lifelog
Annotation & Summarisation
Challenges

ImageCLEF - Lifelog
Annotation & Summarisation
Challenges

171718181919

NTCIR-12 Lifelog
Retrieval and Insights Challenge

NTCIR-13 Lifelog-2
Retrieval, Insights, Segmentation& Annotation Challenge

LSC’18Interactive Retrieval

LSC’19 Interactive Retrieval
NTCIR-14 Lifelog-3
Retrieval, Segmentation andInsights Challenge

ImageCLEF - Lifelog
Annotation & Summarisation
Challenges

2020

2121-22NTCIR-16 Lifelog
Ad-hoc Retrieval Challenge

LSC’20Interactive Retrieval

LSC’21
Interactive Retrieval

Motivation for NTCIR-16
Task•

Increasing relevance to the technology landscape in 2021 and beyond.

•We are noticing increased interest in the retrieval challenge from lifelogs,
especially noticeable by the growing participation in the LSC (LifelogSearch Challenge) participation workshop. A larger potential participantpool.

•Additionally the lifelog retrieval sub-task was always the most popular subtask of our three previous Lifelog tasks at NTCIR 12-14.

•So for NTCIR-16, we are proposing a focused sub-task to address thechallenge of retrieval from an existing lifelog dataset.
•LSAT - Lifelog Semantic Access Task•

Using an existing dataset that is fully anonymised

NTCIR, LSC, and ImageCLEF Lifelog
unique participants (submitting only)Cathal Gurrin (2020)

LSAT - Lifelog Retrieval
The Lifelog Semantic Access (sub) Task in which participants
have to retrieve a number of specific moments in a lifelogger'slife. We define moments as semantic events, or activities that
happened throughout the day. LSAT can be undertaken in aninteractive or automatic manner. We will provide a base-line
retrieval engine (via API) for participants who want to build aninteractive LSAT system.

Example search tasks include:
Find the moment(s) boarding an A380.
Find the moment(s) buying a TV.
Find the moment(s) taking photos of a friendly dog

Data Summary••

Four months of rich lifelog data from one person•

Data requires about 38GB•

With accompanying automatic visual-annotation and metadata•

Location•

Activity•

Biometrics

Data released via individual and organisational agreement (as atNTCIR12-14)

01

Wearable Multimedia

Human Biometrics

1,500 images per day fromthe Narrative wearable
camera. Accompanying
concept annotations.
Periodic audio. Manualphotos captured. Music

24x7 heart rate, galvanic skin
response, calorie burn, steps,
skin temperature. Daily blood
glucose level & bloodpressure. Weekly cholesteroland uric acid measurements.

MINUTE AS
THE UNIT OFRETRIEVAL

030204

Information Access

Human Activity

Onscreen reading, keystrokes
on keyboard, mouse
movements, computer
activity, web pages viewed.

Physical activities (walking,running, transport, etc..),locations visited, food eaten,mood.

</log></food-logs><NTCIR-lifelog-dataset></personal-logs><users><minutes><user id="u1"><minute id="0"><days><location><day><name>Home</name><date>2016-08-15</date></location><image-directory>NTCIR-Lifelog-dataset/u1/2016-08-15/</image-directory><activity>transport</activity><biometrics><bodymetrics><weight>81.89</weight><calories>1.2</calories><fat-mass>19.78</fat-mass><gsr>0.000183189</gsr><heart-rate>67</heart-rate><heart-rate>59</heart-rate><systolic>125</systolic><skin-temp>86.9</skin-temp><diastolic>84</diastolic><steps>0</steps></biometrics></bodymetrics><activities><images><steps>9340</steps><image><distance>8464.468</distance><image-id>u1_2016-08-15_052242_1</image-id><elevation>26.87</elevation><image-path>u1/2016-08-15/20160815_052242_000.jpg</image-path></activities></image><personal-logs></images><health-logs><phone-images><time>06:10</time><phone-image><GLU>6.9</GLU><phone-image-id>u1_2016-08-15_06.23.14_phone_1</phone-image-id><BP>125/84</BP><phone-image-path>u1/2016-08-15/phone/2016-08-15 06.23.14.jpg</phone-image-path><HR>67</HR></phone-image><mood>Anxious</mood></phone-images><comment>Diet change from today..</comment><music></health-logs><song>Runaway Train</song><drink-logs><song-mbid>bb7df441-35b2-4c0d-99b5-6560a5bbbc51</song-mbid><log><artist>Soul Asylum</artist><time>06:20</time><artist-mbid>b10db9ad-b4c3-47f3-a7a4-37864b134f65</artist-mbid><drink>Coffee</drink><album>Black Gold: The Best Of Soul Asylum</album></log><album-mbid>4f2ff67a-d196-48a6-ba0a-bff6724b94ec</album-mbid></drink-logs></music><food-logs></minute><log></minutes><time>06:20</time></day><food>Small fruit breakfast</food></days></user></users></NTCIR-lifelog-dataset>Data & Guide

NTCIR-14

Final task guideline &dataset release

Welcome to Tokyo orco-located eventResults
LAST result & draft
overview paper release

3Mar
22NOW
1

Jun21
Sep21

1

Data Preparation

June
22

2

Jan
22

Topic Release

Submissions

New LSAT topics &baseline system

Participant
submissions withpooled relevancejudgements

2Feb
22

Relevance Judgement Phase

Paper Due
Review process3

Overview Paper͘Π͢;͚ͪͬ͜Δͯ

from:
NTCIR-13 Lifelog Organisation TeamWebsite: ntcir-lifelog.computing.dcu.ie
Contact Email: cathal.gurrin@dcu.ie