AR DeepCalorieCam:AR! # *, ) ( +&"%
$')# : %!-$'
Foodlog
&($.

+," %!-$'
%!- $'*1 :     2

3



[with AR][with ]Multi-task CNN

withAR()[with

 ]

4122

5122



: 

Im2Calories [Myers 2015]
CNN  ",   ,
# !

Myers et al. Im2calories: towards an automated mobilevision food diary. In Proc. of IEEE International
Conference on Computer Vision, 2015.

6!

: 
1(surf, color hist)    0 20
Image-based Calorie
Content Estimation for Dietary Assessment, Workshop on
Multimedia for Cooking and Eating Activities, 20117

821- : $)"!

+9   +&$)"!

(VGG16 [4])%*#(+, ' : Multi-task CNN CNN

 550 kcal
 10 : Multi-task CNN

11550 kcal



CNN#  !
!

" " : 12"#

 83,000 !

5= : 0->:8;UECFOOD100 ?2100
-,(-1!B@%,&/,MGmi

,1ANL

+
miF

13<E$

1'.610074?2- B@I  "1#?2 15 39 4877 HDK) , JC *,
mi

:145}() c

550{

434ka5,l0a43

Word2Vec

Word2Vec,…

4[0.07838 -0.01535 …
0.00202] (500-d)[-0.00599 -0.03586 …
0.01957] (500-d) : CNN    
: 3421
Keras 

http://chainer.org/

: 1456

15 : Multi-task CNN

16      

17: Multi-task CNN/20

R=0.7743%

50%
R=0.8140

76%80%



80%/85%

18: Multi-task CNN 02



0.32
20

35%

R=0.810.81)

R=0.32(()//50%kcal

kcal
02
1

Multi-task CNN(Ours)

02,

19

 : 


315 kcal

419 kcal

85 kcal

493 kcal

315 kcal

429 kcal

89 kcal527 kcal

0 kcal-10 kcal-4 kcal-34 kcal
 : 



20

323 kcal

553 kcal

330 kcal

58 kcal

670 kcal

838 kcal

148 kcal

115 kcal-347 kcal-285 kcal

182 kcal-57 kcal 



!
!





21




[with AR][with ]Multi-task CNN
DeepCalorieCam

AR DeepCalorieCam

withAR()[with ]AR DeepCalorieCam V2





22





[with AR][with ]Multi-task CNN
DeepCalorieCam

AR DeepCalorieCamwithAR()[with ]AR DeepCalorieCam V2! : #

- &/" (*$)23

1. YOLOv2 - &
2. &)- ,+ '% (0
3. )- (0./" CNN !#Food Detection

YOLOv2
Calorie Estimation
518 kcal

Inceptionv3 "  

24

(20179)25…

Chainer2C(Caffe2C)
Chainer/Caffe$iOS/AndroidC! %"&#  →CCPU

Caffe2C
Chainer2C

Your App
Chainer2C
Accelarete (Blas)
CPUCoreMLiOS

coremltools

mxnet-tocoreml

torch2coreml

deployCore ML modelCore MLYour App

26



()

Tanno, R., Ege, T. and Yanai, K.: DeepCalorieCam: An iOS App for Dish
Detection and Calorie Estimation, Proc. of International Workshop on
Multimedia Assisted Dietary Management (MADIMA) (2017)27

AR

28

29

11RR

1R

1 2A RR

(%$30&

+
#"
!
,
!
", "'  ) *-&       31Canny 
      


32
k-Means
k=3, ,      

33


 GrabCut       


34





35



   GrabCut GrabCut36
 GrabCut
   

46.8!" #180.5!" #37

       : 43149
 : 46.8!"# ()   : 68066
 : ?? !"#38


 : 68066: ?? 123
a=0.001,b=0.5,c=37

  : 43149: 46.8123&'!" = $" ∗ ( , * = + ∗ !", + . ∗ !" + /'

: !" ,  : !0

: 1.577: * ,  : +, ., /: $" ,  : $0

: 73.8 /4, 

: 74.4 kcal

39AR




40






 !
#
"$
 

/0 : 89 *512(+AR 5()
6# +)FoodCam ,3 $(7&, AR
%,3'1) 314"
!.2- : :; ,  , 41



FoodCam (2015 )42 0.5~1.5 

43.(5+  20 "/  $- & )   *'#%,46962
552
14

FoodCam
3+ .( 3+ .(!1 5+ !1 5+-53.25 209.79 -242 55.10-242 91.26 -47.08 52.52
54.83 36.28 4.86 11.87

AR
3+ .(!1 5+-67.14
18.8-127.0
9.0-0.95
0.16

44



().( )#$4-31 !,5 +6'2 -> 0AR4-31 !,5 '228&"
4*9%7/



AR

.-  )*-'01$/,+( "-'3! *$/&%

"

#245

462RR

i 2- -

R1l a

R2lCutCC-> C
a 1 CsAaNMk247