Package: emblaze
Emblaze - Interactive Embedding ComparisonEmblaze is a Jupyter notebook widget forvisually comparing embeddingsusing animated scatter plots. It bundles an easy-to-use Python API for performing dimensionality reduction on multiple sets of embedding data (including aligning the results for easier comparison), and a full-featured interactive platform for probing and comparing embeddings that runs within a Jupyter notebook cell.Read the documentation >InstallationCompatibility Note:Note that this widget has been tested using Python >= 3.7. If you are using JupyterLab, please make sure you are running version 3.0 or higher. The widget currently does not support displaying in the VS Code interactive notebook environment.Install Emblaze usingpip:pipinstallemblazeThe widget should work out of the box when you runjupyter lab(see example code below).Jupyter Notebook note:If you are using Jupyter Notebook 5.2 or earlier, you may also need to enable
the nbextension:jupyternbextensionenable--py--sys-prefixemblazeExamplesPlease seeexamples/example.ipynbto try using the Emblaze widget on the Boston housing prices or MNIST (TensorFlow import required) datasets.Example 1: Multiple projections of the same embedding dataset.This can reveal areas of variation in the dimensionality reduction process, since tSNE and UMAP are randomized algorithms.importemblazefromemblaze.utilsimportField,ProjectionTechnique# X is an n x k array, Y is a length-n arrayX,Y=...# Represent the high-dimensional embeddingemb=emblaze.Embedding({Field.POSITION:X,Field.COLOR:Y})# Compute nearest neighbors in the high-D space (for display)emb.compute_neighbors(metric='cosine')# Generate UMAP 2D representations - you can pass UMAP parameters to project()variants=emblaze.EmbeddingSet([emb.project(method=ProjectionTechnique.UMAP)for_inrange(10)])# Compute neighbors again (to indicate that we want to compare projections)variants.compute_neighbors(metric='euclidean')w=emblaze.Viewer(embeddings=variants)wExample 2: Multiple embeddings of the same data from different models.This is useful to see how different models embed data differently.# Xs is a list of n x k arrays corresponding to different embedding spacesXs=...# Y is a length-n array of labels for color-codingY=...# List of strings representing the name of each embedding space (e.g.# "Google News", "Wikipedia", "Twitter"). Omit to use generic namesembedding_names=[...]# Make high-dimensional embedding objectsembeddings=emblaze.EmbeddingSet([emblaze.Embedding({Field.POSITION:X,Field.COLOR:Y},label=emb_name)forX,emb_nameinzip(Xs,embedding_names)])embeddings.compute_neighbors(metric='cosine')# Make aligned UMAPreduced=embeddings.project(method=ProjectionTechnique.ALIGNED_UMAP)w=emblaze.Viewer(embeddings=reduced)wExample 3: Visualizing image data with image thumbnails.The viewer will display image previews for each point as well as its nearest neighbors. (For text data, you can useTextThumbnailsto show small pieces of text next to the points.)# images is an n x 100 x 100 x 3 numpy array of 100x100 RGB images (values from 0-255)images=...thumbnails=emblaze.ImageThumbnails(images)w=emblaze.Viewer(embeddings=embeddings,thumbnails=thumbnails)wYou can also visualize embeddings with multimodal labels (i.e. where some points have text labels and others have image labels) by initializing anemblaze.CombinedThumbnailsinstance with a list of otherThumbnailsobjects to combine.See thedocumentationfor more details on defining and configuring comparisons with Emblaze.Development InstallationClone repository, then install dependencies:pipinstall-rrequirements.txtInstall the python package. This will also build the JS packages.pipinstall-e.Run the following commands if you useJupyter Lab:jupyter labextension install @jupyter-widgets/jupyterlab-manager --no-build
jupyter labextension install .Run the following commands if you useJupyter Notebook:jupyter nbextension install --sys-prefix --symlink --overwrite --py emblaze
jupyter nbextension enable --sys-prefix --py emblazeNote that the--symlinkflag doesn't work on Windows, so you will here have to run
theinstallcommand every time that you rebuild your extension. For certain installations
you might also need another flag instead of--sys-prefix, but we won't cover the meaning
of those flags here.How to see your changesOpen JupyterLab in watch mode withjupyter lab --watch. Then, in a separate terminal, watch the source directory for changes withnpm run watch. After a change to the JavaScript code, you will wait for the build to finish, then refresh your browser. After changing in Python code, you will need to restart the notebook kernel to see your changes take effect.Standalone App DevelopmentTo develop using the standalone app, runnpm run watch:standalonein a separate terminal from the Flask server to continuously build the frontend. You will need to reload the page to see your changes.The standalone application serves datasets stored at the data path that is printed when the Flask server starts (should be something like.../lib/python3.9/site-packages/emblaze/datafor the pip-installed version, or.../emblaze/emblaze/datafor a local repository). You can add your own datasets by building anEmbeddingSetand (optionally) aThumbnailsobject, then saving the results to files in the data directory:importos,jsondataset_name="my-dataset"data_dir=...# data directory printed by flask serverembeddings=...# EmbeddingSet objectthumbnails=...# (Text|Image)Thumbnails objectos.mkdir(os.path.join(data_dir,dataset_name))withopen(os.path.join(data_dir,dataset_name,"data.json"),"w")asfile:json.dump(embeddings.to_json(),file)withopen(os.path.join(data_dir,dataset_name,"thumbnails.json"),"w")asfile:json.dump(thumbnails.to_json(),file)Building DocumentationInstall pdoc3:pip install pdoc3Build documentation:pdoc--html--force--output-dirdocs--template-dirdocs/templatesemblazeDeploymentFirst clean all npm build intermediates:npm run cleanBump the widget version inemblaze/_version.py,emblaze/_frontend.py, andpackage.jsonif applicable. Then build the notebook widgets and standalone app:npm run build:allRun the packaging script to generate the wheel for distribution:python -m buildUpload to PyPI (replace<VERSION>with the version number):twine upload dist/emblaze-<VERSION>*Development NotesSvelte transitions don't seem to work well as they force an expensive re-layout operation. Avoid using them during interactions.