Analyzing Regional Food Trends
with Geo-tagged Twitter Food Photos
Kaimu Okamoto
Keiji Yanai
The University of Electro-Communications, Tokyo
3-2.EXPERIMENTS[2]：Trend Analysis

1. INTRODUCTION
Regional differences are observed in the photos posted to Twitter,because peoples’ lifestyles differ from region to region.
• Since foods are essential to human life, the regional differences areexpected to be larger.
However, the regional difference on food images over Twitter has
not been explored so far.

We analyze regional food trends !

2. METHOD[1] Classifying food and non-food photos
Fine-tuning ResNet to prepare food/non-food classifier
To select only food images from raw Twitter photos
Food images• UECFOOD100[2]• Food101[3]Non-Food images• ILSVRC2012 ImageNet• Kawano et al [4] images

South-EastAsia
North
America

29.3

12.5 %

Noodles16.6 %

14.8

14.5

5.8 5.8

Rice

Fried food Sea food

10.8

10.2

SoupMeat12.6 %

10.3

18.6

15.4

8.4

SaladSouthAmerica

6.7 %Europe10.4 %

25.1

22.5

Sweets

11.6

10.0

Fast food

Bread

17.6

18.7

12.1

11.0

Beverage
11.8 %

23.2

17.7

22.26.9Classify food andnon-food[2] Extracting food CNN features
Training of new CNN features by fine-tuning VGG16 with Triplet & cls. loss
To extract food-specialized features which can discriminate small
differences
Triplet loss
positive
negative
Food images• UECFOOD100[2]• Food101 [3]‒ East Asia：“Noodles” and “Rice”, which are relatively rare in the otherregions, are included at the top.
‒ South-East Asia：“Soup” is ranked at the top, and many dishes are made ofvegetables and meats in soup.
‒ North America, South America, Europe：It turned out that 4 items of thetop 5 items of the categories are the same
‒ Middle East：The top five food categories are the same as Europe. However
there are many brown coffee photos which are a unique type to Middle East.

Region Trend Analysis by Food
Noodles＋81.0 %

12.9

3.4

East AsiaClassificationloss

𝐿 = 𝐿 𝑇 + 𝐿𝐶

East Asia

Middle East

Test image classify
97.8%

The entire loss

Food Trend Analysis by Region

SaladK-means clustering for the food images using the food CNN featuresclassification cluster of the image into 17 food represent categories
121.235.1North America

Soup[3] Clustering and analyzing of regional tendency

16.2 %
14.7 %

66.3
South-East Asia

8.4

‒ Noodles：East Asia’s ratio is the highest, many “ramen” noodles images areshown‒ Salad：North America’s ratio is the highest‒ Soup：East-South Asia’s ratio is the highest, using other categories of food
Noodles in East Asia

Salad in North America Soup in East-South Asia

[4] Visualization of regional parts• We find out which parts of the food images correspond to regionalfeatures by Grad-CAM [1].
• To visualize regional features, we train region classifiers on some of the
pre-defined food categories.

3-3.EXPERIMENTS[3]: Visualization of Regional Features
Example of clear regional features

3-1.EXPERIMENTS[1]: Twitter Geotagged Food Images
Selecting food images from Twitter images in 2016 for whole a year with the
food/non-food classifier
190,000 food images from 3.78 million raw Twitter images
900,000
800,000
700,000
600,000

all images
food images

1.8%

3.0%
9.8%

14.3%

2.0%

500,000

Excluding “SouthAsia”, ”Africa, “Oceania”
from analysis because
the number of foodimages are not enoughMeat in South-East Asia : Ball parts

Meat in Middle East : Vegetable parts

Meat in East Asia：Skewer parts

Meat in North America：Flat parts

400,000
300,000
200,000
100,000

４.CONCLUSIONS AND FUTURE WORK

2.7%
1.2%

0.8%

0.8%

0East Asia South Asia South-East North
Asia
America

Analysis target regions

SouthAmerica

EuropeAfricaOceania

MiddleEast• We have analyzed regional food tendency on only the geotagged food images without using any textual information.
• Integration image data of the areas that could not be analyzed
from the other photo SNSs such as Instagram and Weibo.
[1] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-CAM: Visual explanations
from deep networks via gradient-based localization. In CVPR, 2016.
[2] Y. Matsuda, H. Hoashi, and K. Yanai. Recognition of multiplefood images by detecting candidate
regions. In ICME, 2012.
[3] B. Lukas, G. Matthieu, and V.G. Luc. Food-101 – mining discriminative components with random forests.
In ECCV, 2014.
[4] Y. Kawano and K. Yanai. Automatic expansion of a food image dataset leveraging existing categories
with domain adaptation. In ECCVWS, 2014