Vector tile: Introduction to UNVT tools ( a kind of a tool catalog (not endorsed). As of April 2022)　

The original article was written in Japanese ([here](https://qiita.com/T-ubu/items/3f1a431fdd1e39d07ade)) in April 2022. This article was translated and released in English in May 2022.

# Introduction
The United Nations Vector Tile Toolkit (UNVT) was launched by @hfu in 2018 and is promoted under the UN Open GIS Initative. So far, various tools have been developed, which we can use them in each step of vector tile processing such as production, styling, hosting, optimization, and consumption (utilization).

In this article, I would like to introduce some of the UNVT tools that I use most often. **I hope that it will help newbies of UNVT to have some idea on what tools are available**. In addition, for those who work for vector tiles, I hope this article **will serve as a kind of a catalog to show what UNVT can do.** 

# Disclaimer
This article is based on my own experience as of April 2022. This article was first drafted in Japanese ([here](https://qiita.com/T-ubu/items/3f1a431fdd1e39d07ade)), then translated into English.

I am going to introduce each tool with its expected work area (production, styling, hosting, optimization, consumption, etc.), but it depends on my personal experiences and they may be used in different fields.

UNVT uses a lot of open source tools, so the tools introduced here should be called as a group of tools, and there are some overlaps among them. But I simply use "tool" in this article to simplify my explanation.

# Tool list
## Tools I use most often (They can be used for various purposes) 
| Tool name | Release date (month) | Main Developer(s) | Areas of use | Overview |
|:-|:-: |:-: |:-: |:-|
| unvt/tell | 2021.9 | hfu | Use | You can create a storytelling map using a vector tile map in the background (MapLibre library). As you scroll through the home page, the map scene moves and explanation box appears. Stories can be written in YAML files. |
| unvt/charites | 2021.10 | Geolonia Inc. | Styles | Vector map styles based on MapBox (and MapLibre) style specifications can be created/edited using easy-to-edit YAML files. We can also import style from a json style style file to YAML file, and vice versa. A live map preview function enable us to check our edits on the fly. A tool donated by Geolonia to UNVT. |
| unvt/kata | 2022.3 | GSI, Japan, Geolonia Inc. | Production, Optimization | It has the "filter" function to add layer name, maximum zoom level, minimum zoom level to each source file and output them to JSON sequence to be converted into vector tile. There is a "document" function that shows the layer name, attribute name, and zoom level range from the vector tile metadata. By passing the filtered data to Tippecanoe, you can easily create a vector tile with adjusted layer structure. |
| unvt/itoma | 2022.3 | GSI, Japan, Geolonia Inc. | Styles, Hosting | It displays vector tile map styles (YAML format imported by charites) in multiple vector tile libraries (Mapbox GL JS, MapLibre GL JS, ArcGIS API for Javascript) ) as preview by localhost. In addition, since it returns a response based on the VectorTileServer service of the ArcGIS REST API, you can check the use of vector tile maps in ArcGIS Online. (Note: Localhost needs to be https) |
| unvt / nanban | 2020.4 | hfu | General | Ubuntu / Intel based UNVT docker files. Give Windows users a Linux environment through a Docker container. |
| unvt / equinox | 2020.3 | hfu | General | UNVT related tool installer for Raspberry Pi. |

There are more tools undre UNVT although they are not written here. For example, UNVT Portal is one of the tools and it was reported to UN Open GIS monthly meeting. I hope I can introduce them somewhere.)

## Tools for advanced users (The usage will become a little more specialized) 

| Tool name | Release date | Developer (main) | Areas of use | Overview |
|:-|:-: |:-: |:-: |:-|
| unvt/onyx | 2019 | hfu | Host | A simple and scalable vector tile server. It is a nodejs/express server to host vector tiles and it delivers vector tiles in pbf format from mbtiles files. It can be executed in PM2. |
| unvt/marble | 2021.12 | ubukawa | Hosting | A server that extends the above mentioned onyx to deploy a function to return a response that conforms to the ArcGIS REST API VectorTileServer. It works with nodejs/express. |
| unvt/naru | 2020 | hfu | General (Capacity Building) | It works with Raspberry Pi and Docker. After downloading OSM, you can convert it to vector tiles, create styles using hocon, prepare sprites, and so on. It is a material for capacity building. |
| unvt/nsft | 2022.2 | ubukawa | Hosting | This is a workplace where I converted Noto Sans series fonts to PBF text fonts. It can be applicable to other fonts. |
| unvt/oz | 2021.9 | hfu | Hosting | A tool for server-side overzooming. Create large zoom level tiles from existing files. |
| unvt/zfxy-spec | 2022.3 | hfu | General | This is a draft specification for expanding zxy vector tile schema. It introduces the height index "f" to define a 3D tile space. We are paying attention to future development.


# Tool introduction / Details (Selected tools from the above list)
I won't cover all of the tools above, but I will introduce some detailes of the selected tools. If you have any question or request about our tools, please let me know in the comments. I will reply as much as possible. 

## unvt/tell
### overview:
You can create a storytelling map with a vector tile map in the background (MapLibre). As you scroll through the home page, the map scene moves and explanations appear. Stories can be written in YAML files.

* You can create a map story by preparing an html file and a YAML file that specifies the location of the story and map. You can also use YAML tags in the html file.
* This is a script developed as a workshop material and developed based on the scripts at https://github.com/optgeo/s (developed by hfu, taisukef, halsk).
* This storytelling related work was inspired by mapbox, and developers explicitly express their appreciations to mapbox. thanks.

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/679143/aab7f295-9a3a-8e49-6779-069a3856fb61.png)
 
### Repository URL:
https://github.com/unvt/tell

### Past efforts and references:
* UNVT Storytelling workshop (2022-10-22):
    * Recording: https://www.youtube.com/watch?v=CVajhAUDLMs
    * Slide: https://speakerdeck.com/hfu/unvt-storytelling
* My past blog: https://qiita.com/T-ubu/items/b970339a49d8f584acf6

## unvt/charites
### overview:
Vector map styles based on MapBox (and MapLibre) style specifications can be created/edited using easy-to-edit YAML files. We can also import style from a json style style file to YAML file, and vice versa. A live map preview function enable us to check our edits on the fly. A tool donated by Geolonia to UNVT. **(I'm a Windows user, so I often use this tool with unvt/nanban (a docker container).)**

* Create a YAML-formatted style from scratch, or import an existing style (JSON) into YAML format.
* You can edit the style in YAML format.
* Manage style in YAML files and classifies layers into groups to improve maintenance convenience and style readability.
* Edited YAML files can be output in JSON format.
* Real-time preview is also available, so you can see the result of your edit on the fly.

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/679143/f593642f-4607-f382-4642-de057186bb36.png)
(As shown in this figure, one style information is managed by multiple YAML files.)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/679143/94c377f5-881e-2a70-df44-b2d5cf41db7c.png)
(Edit the YAML file while doing a live preview.)


### Repository URL:
https://github.com/unvt/charites
 
### Past efforts and references:
* Geolonia's blog: https://blog.geolonia.com/2021/10/21/unvt-charites.html
* My past articles
    * How to edit / modify a style.json of your vector tile map using UNVT / charites: https://qiita.com/T-ubu/items/e6794b81e45dea34548c
    * Process the style of GSI Map Vector (tentative name) with unvt / charites and see it in ArcGIS Online (first step): https://qiita.com/T-ubu/items/87b17bb8752cb15c9d28
    * How to use UNVT styling tool, charites, to edit a style json from ArcGIS server https://qiita.com/T-ubu/items/6e31a6bc5a458b91d4cd

## unvt/kata
### overview:
It has the "filter" function to add layer name, maximum zoom level, minimum zoom level to each source file and output them to JSON sequence to be converted into vector tile. There is a "document" function that shows the layer name, attribute name, and zoom level range from the vector tile metadata. By passing the filtered data to Tippecanoe, you can easily create a vector tile with adjusted layer structure. **(I'm a Windows user, so I often use this tool with unvt / nanban for some purposes.)**

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/679143/5894e19f-920d-1ed8-8d43-05ad0ab21e79.png)
(The figure is from the GitHub repository. Although it is an initial concept, you can understand that ① is the document function and ② is the filter function.)

* unvt/kata can easily give the layer name and maximum / minimum zoom level for each source by using the filter function when creating vector tiles. You can easily create a vector tile by passing the filtered one to tippecanoe with the following command.
```
kata filter kata.yml | tippecanoe -e zxy2 --no-tile-compression --no-feature-limit --no-tile-size-limit
```
* I think it is a very useful tool because you can check the layer list and property list of vector tiles.
* If the data exceeds several GB, there is no setting to divide the output file according to the region, so I think that something other than this tool is needed, but it perfectly fits for small data conversion.

### Repository URL:
https://github.com/unvt/kata

### Past efforts and references:
* My past articles
    * Making Vector Tile from GeoJSON using unvt (unvt/kata and unvt/nanban) https://qiita.com/T-ubu/items/4fa0ab633c6746574bdd
    * Try using unvt/kata (0th) https://qiita.com/T-ubu/items/22b6c4c07fca9c754df6 (* This article has not completed all the work due to a bug in the old version. Now This issue has been fixed in v1.0.4 of.)

## unvt/itoma
### overview:
It displays vector tile map styles (YAML format imported by charites) in multiple vector tile libraries (Mapbox GL JS, MapLibre GL JS, ArcGIS API for Javascript) ) as preview by localhost. In addition, since it returns a response based on the VectorTileServer service of the ArcGIS REST API, you can check the use of vector tile maps in ArcGIS Online. (Note: Localhost needs to be https) **(I'm a Windows user, so I often use this tool with unvt / nanban for some purposes.)**

* You can see a preview of the style from your local host.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/679143/0581ed9f-ab4e-fe29-9876-02bd6f063a66.png)
(↑ You can preview in 3 libraries.)
* Since it is also previewed in ArcGIS Online, it returns a response according to the ArcGIS REST API. By executing "itoma serve", you can check the response of ArcGIS REST API from http://localhost:8080/arcgis/rest/services/{any_name}/VectorTileServer.
    * index.json: http://localhost:8080/arcgis/rest/services/{any_name}/VectorTileServer/index.json
    * root.json (style): http://localhost:8080/arcgis/rest/services/{any_name}/VectorTileServer/resources/styles/root.json
    * sprites: http://localhost:8080/arcgis/rest/services/{any_name}/VectorTileServer/resources/sprites/
    * glyphs (fonts): http://localhost:8080/arcgis/rest/services/{any_name}/VectorTileServer/resources/fonts/　
    * tilemap: http://localhost:8080/arcgis/rest/services/{any_name}/VectorTileServer/tilemap (However, tilemap in this tool  uses a fixed values ​​for pre-determined zoom levels. When you use your own tiles, you may need to edit the source codes.)
* Actually, in order to see the vector tile map in ArcGIS Oline, you need to see the localhost with https protocol, and you need to set CORS. (We can so so. You can see my past experience in the other blog.)

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/679143/e23d3c55-b53a-962d-9a3e-0ca50a6706c0.png)

### Repository URL:
https://github.com/unvt/itoma

### Past efforts and references:
* My past articles
    * itoma (1): Try the ArcGIS Rest API implementation tool unvt/itoma for vector tiles https://qiita.com/T-ubu/items/f40f6dd23724cae92db4
    * itoma (2): Make unvt/itoma (ArcGIS REST API implementation tool for vector tiles) https-enabled https://qiita.com/T-ubu/items/97495f40c8477515239f

## unvt/nanban
### overview:
Ubuntu/Intel based UNVT docker file. Even Windows users can get a Linux-like working environment through the Docker container, so vector tile conversion using tippecanoe can be done easily. Since gdal (ogr2ogr), nodejs, ruby, etc. are also included, it can be used not only for vector tile creation but also for hosting test and style test.

The main components of this tool are:

* nodejs
* gdal-bin
* ruby
* sqlite3
* tmux
* vim
* tarn
* mapbox/tippecanoe
* budo
* hjson
* pm2
* phshcorn/hocon-parser
* vt-optimizer (It doesn't work with recent nodejs, so I haven't put it in the current nanban.)
* unvt/charites
* unvt/kata
* unvt/itoma

Please read carefully as the policy regarding the use of Docker changed in the summer of 2021 ([Docker FAQ] (https://www.docker.com/pricing/faq/)).

> Docker Desktop may be used for free as part of a Docker Personal subscription for:
..
Non-commercial open source projects


### Repository URL:
https://github.com/unvt/nanban

https://hub.docker.com/r/unvt/nanban

### Past efforts and references:
* My past articles
    * Create vector tiles with Docker (unvt/nanban) (for Windows users) https://qiita.com/T-ubu/items/76a60bfd4a27d11521e2

## unvt/equinox
### overview:
UNVT related tool installer for Raspberry Pi. You can install tools related to vector tile processing by curling and running the installer from Raspberry Pi.

As for the component of the tool, I will not write it in detail here because it is easy to understand by looking at @hfu's article. In addition to the general tools such as nginx, nodejs and ruby, it also contains geospatial and vector tile related tools like Tippecanoe, gdal, pdal, charites, vt-optimizer.

A few years ago, we often run a Docker container on a Raspberry Pi, but I think that by using this equinox, it is possible to build a stable environment without using a Docker image.


### Repository URL:
https://github.com/unvt/equinox

### Past efforts and references:
* unvt/equinox component review (hfu, 2021)
 https://qiita.com/hfu/items/8792cf354faebf9b0efb
* UNVT Guidance for those who want to move their hands early 2020 (hfu, 2020)
 https://qiita.com/hfu/items/4194c3b35b76d0a696ee
* Create a vector tile server with Raspberry Pi (T-ubu, 2021) https://qiita.com/T-ubu/items/f5cb639b4c337ef1050d

## Acknowledgments
From the beginning to the present, @hfu's contributio is wonderful. Thank you for your daily efforts. We have also been happy to have a large number of partners as the UNVT community. In particular, I think the fact that Georgia donated unvt/charites is a big step forward. Thank you.
We would like to thank all the university students for participating in the UNVT hackathon in December 2021. I am also looking forward to seeing what progress will be made in the future.

## References
(For reference of each tool, citations and links are summarized in the text.)