21/October/2022

Daylight Earth Tables

Jennings Anderson, Jonah Adkins, Jacob Wasserman

Maps atMeta

OpenStreetMap

19/October/2022

Daylight Map Distribution

Earth Table

Maps in Meta Products
Instagram
FacebookMarketplace
And more…
2map data © OpenStreetMap

Daylight Earth Table

Overview

What is the Daylight Map Distribution?
What is the Earth Table?
Introducing: Daylight Earth Table
Working with the Daylight Earth Table

3

Daylight Earth Table

Daylight

A monthly distribution of OpenStreetMap that undergoes a series ofquality control and vandalism checks to ensure a degree of map quality andintegrity.

Daylight Earth Table

Timeline of Daylight
OpenStreetMap Distribution< 20192020

2021

2022

Today

State of the Map
2018 & 2019

March 2020

April 2021

January 2022

October 2022

Announcing
Daylight v0.1

Daylight v1.0
released

Daylight published
on AWS Registry ofOpen Data as PBF
and Analysis-Readyparquet files

Daylight v1.18

Mobius LogicalChangesets(LoChas)https://daylightmap.org

5Daylight Earth Table

Daylight Process:Find-Fix-Import Loop
Find: Discover errors or other issues anywhere on the map
Fix: Submit fixes on live OSM, not in an internal database
Import: Apply fixes from OSM into the Daylight map

6Daylight Earth Table

Daylight v1.18

525 M

78 M

100%

Buildings

Kilometers of roads / paths

OpenStreetMap Data
Think of Daylight as a snapshotof OSM where each featuremight not be from the samesnapshot.

7

Example: Repaired beach relation for Padang Bai Beach in Bali, Indonesia

Daylight Earth Table

Daylight v1.18
Changelog andsummaries of fixes
made to OSM available
on daylightmap.org

8Daylight Earth Table

Where can I find Daylight?
-OSM PBF Format-Analysis-ReadyCloud-Optimized
Parquet Files

daylightmap.org

registry.opendata.aws/daylight-osm/

9Daylight Earth Table

Earth Table

Earth Table Schema

Where did the Earth Table come from?
-Growing internal use cases toquery the earth - “I need all theparks in the world to do x”-

Using basemaps is across-functional tool for projects
across the org.

-

Needs to be simple - tagging
complexities of OSM are a highbarrier to entry-

Needs to be in a single table, witheasy to understand schema &hierarchy
11

Earth Table Schema

Before:Cartography team did all of the OSM tag interpretation and created
map tiles.
Needed to look inside tiles to find well-formatted, translated map data

Cartography
Ingestion,Cleaning,Validation, etc.

Tag Interpretation

Map Tiles

Map Surfaces

After:The earth table is created by interpreting OSM tags into a simplified
3-level ontological schema.
Anyone can access this table, especially cartography

Cartography
Ingestion,Cleaning,Validation, etc.

Tag Interpretation

EarthTable

Map Tiles

Map Surfaces
12Earth Table Schema

It doesn’t solve everything…

13

map data © OpenStreetMap

Daylight Earth Table

14

map data © OpenStreetMap

Earth Table Schema

Cartographic Data Improvements:
building

525,240,408

building_detail●land theme includes processed global coastlines

2,232,081●

building height information is validated and normalized

infrastructure

11,226,443●

placename theme includes simplified classification -> all places

land

40,831,400

landuse39,397,340●

lengths and areas calculated as attribute columns

placename3,430,404●

Bing Tile Quadkey applied to each featurepoi

38,777,950●

Lots of boolean evaluations for easy filtering:road

216,890,017

transit

6,716,006

water

36,401,694

grouped into 3 classes: urban, settlement, local“Is_indoor”, “is_intermittent”, “is_bridge”, “is_area”, etc●

Buildings include “landuse” class they are within●

Building_detail includes building id they are part of15

Earth Table Schema

83% Earth Table Data Source
16% Internal Data Sources that are “Earthified”
1% Curated Cartographic Map Features16

map data © OpenStreetMap

Earth Table Schema17

map data © OpenStreetMap

Earth Table Schema18

map data © OpenStreetMap

Earth Table Schema

19

map data © OpenStreetMap

Daylight Earth Table

Accessing Daylight EarthTabless3://daylight-openstreetmap/earth
20

Accessing Daylight Earth Tables

Daylight Earth Table

21

daylightmap.org/earth

Accessing Daylight Earth Tables

Daylight Earth Table

Using Amazon Athena

CREATE EXTERNAL TABLE `daylight_earth`(`geometry_id` string,`class` string,`subclass` string,`metadata` string,`original_source_tags` string,`names` string,`quadkey` string,`wkt` string )PARTITIONED BY (`release` varchar(5),`theme` string )STORED AS PARQUET
LOCATION's3://daylight-openstreetmap/earth' TBLPROPERTIES ('has_encrypted_data'='false','parquet.compression'='GZIP')

22

daylightmap.org/earth

Accessing Daylight Earth Tables

Daylight Earth Table

23

Accessing Daylight Earth Tables

Daylight Earth Table

24

Accessing Daylight Earth Tables

Daylight Earth Table

25

Accessing Daylight Earth Tables

Earth Table Metadata

Normalize attributes intometadata such as height
Append additional
information from otherthemes for overlapping
featuresearth_tablesource: osm
theme: buildingclass: industrial
subclass: warehouseheight: 20.52 meters
area: 4962 square meters
landuse class: developed,
landuse subclass: industrial
wkt: POLYGON ((26.483607 ….))

26

Accessing Daylight Earth Table

Daylight Earth Table

27

daylightmap.org/earth/landuse

Accessing Daylight Earth Table

Daylight Earth Table28

map data © OpenStreetMap

Accessing Daylight Earth Table

Daylight Earth Table29

map data © OpenStreetMap

Accessing Daylight Earth Table

Daylight Earth Table30

map data © OpenStreetMap

Accessing Daylight Earth Table

Daylight Earth Table31

map data © OpenStreetMap

21/August/2022

Thank youdaylightmap.org/earth
daylghtmap.orgjenningsa@meta.com
jonahadkins@meta.com