Repository: pytorch
Issue: Saving embedding vectors as TSV for Tensorboard is not efficient
Labels: oncall: visualization

## 🐛 Bug

When `add_embedding()` of `tensorboard.SummaryWriter` is called, the embeddings are saved as TSV.
However, this is not efficient in terms of disk space and io time.
Saving as npy is a possible alternative to resolve this because Tensorboard supports loading npy as embedding vectors.

## To Reproduce

Save some embedding vectors like below:

```python
import torch
from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter()
writer.add_embedding(torch.randn(10000, 100, dtype=torch.float32))
```

This code generates TSV file of the embedding at `./runs/SOME_RANDOM_STR/00000/default/tensors.tsv`.
The size of the file is about 19MB.


## Expected behavior

Expected size is about 10000 x 100 x 4 (bytes) = 4MB.
If the embedding is saved as npy, the size will be around 4MB.

Moreover, the saving procedure is slow maybe because every single embedding is parsed by a `for` loop.
https://github.com/pytorch/pytorch/blob/40c0fffb4bb4a7d49cb50c6b276565aa60b3daaf/torch/utils/tensorboard/_embedding.py#L68-L70
It can be slower than save as a single npy file by `numpy.save()`.

## Environment
Using `pytorch/pytorch:1.7.1-cuda11.0-cudnn8-devel` from DockerHub.

```
PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: 11.0
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: Could not collect

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 440.95.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.4
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] pytorch-lightning==1.1.6
[pip3] torch==1.7.1
[pip3] torchelastic==0.2.1
[pip3] torchvision==0.8.2
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.0.221             h6bb024c_0
[conda] mkl                       2020.2                      256
[conda] mkl-service               2.3.0            py38he904b0f_0
[conda] mkl_fft                   1.2.0            py38h23d657b_0
[conda] mkl_random                1.1.1            py38h0573a6f_0
[conda] numpy                     1.19.2           py38h54aff64_0
[conda] numpy-base                1.19.2           py38hfa32c7d_0
[conda] pytorch                   1.7.1           py3.8_cuda11.0.221_cudnn8.0.5_0    pytorch
[conda] pytorch-lightning         1.1.6                    pypi_0    pypi
[conda] torchelastic              0.2.1                    pypi_0    pypi
[conda] torchvision               0.8.2                py38_cu110    pytorch
```