Package: embcompare
EmbCompareA simple python tool for embedding comparisonEmbCompare is a small python package highly inspired by theEmbedding Comparator toolthat helps you compare your embeddings both visually and numerically.Key features :Visual comparison: GUI for comparison of two embeddingsNumerical comparison: Calculation of comparison indicators between two embeddings for monitoring purposesEmbCompare keeps things simples. All computations are made in memory and the package does not bring any embedding storage management.If you need a tool to store, compare and track your experiments, you may like thevectoryproject.Table of contentTable of content🛠️ Installation👩‍💻 UsageConfig fileJSON comparison report generationGUI🐍 Python APIEmbeddingEmbeddingComparisonJSON reportsEmbeddingReportEmbeddingComparisonReport📊 Create your custom streamlit app🛠️ Installation# basic installpipinstallembcompare# installation with the gui toolpipinstallembcompare[gui]👩‍💻 UsageEmbCompare provides a CLI with three sub-commands :embcompare addis used to create or update a yaml file containing all embeddings infos : path, format, labels, term-frequencies, ... ;embcompare reportis used to generate json reports containing comparison metrics ;embcompare guiis used to start astreamlitwebapp to compare your embeddings visually.Config fileEmbCompare use a yaml file for referencing embeddings and relevant informations. By default, EmbCompare is looking
for a file named embcompare.yaml in the current working directory.embeddings:first_embedding:name:My first embeddingpath:/abspath/to/firstembedding.jsonformat:jsonfrequencies:/abspath/to/freqs.jsonfrequencies_format:jsonlabels:/abspath/to/labels.pkllabels_format:pklsecond_embedding:name:My second embeddingpath:/abspath/to/secondembedding.jsonformat:word2vecfrequencies:/abspath/to/freqs.pklfrequencies_format:pkllabels:/abspath/to/labels.jsonlabels_format:jsonTheembcompare addcommand allow to update this file programatically (and even create it if it does not exist).JSON comparison report generationEmbCompare aims to help to compare embedding thanks to numerical metrics that can be used to check if a new
generated embedding is very different from the last one. The commandembcompare reportcan be used in two ways :With a single embedding. In this case it generate a small report about the embedding :embcomparereportfirst_embedding# creates a first_embedding_report.json file containing some infos about the embeddingWith two embeddings. In this case it generate a comparison report about the two embeddings :embcomparereportfirst_embeddingsecond_embedding# creates a first_embedding_second_embedding_report.json file containing comparison metricsGUIThe GUI is also very handy to compare embeddings. To start the GUI, use the commandeembcompare gui.
It will launch a streamlit app that will allow you to visually compare the embeddings you added in the configuration file.🐍 Python APIEmbCompare provide several classes to load and compare embeddings.EmbeddingTheEmbeddingclass is child of thegensim.KeyedVectorsclass.It add few functionalities :You can provide term frequencies so you can filter the elements laterYou can easily compute all elements nearest neighbors (thanks tosklearn.neighbors.NearestNeighbors)importjsonimportgensim.downloaderasapifromembcompareimportEmbeddingword_vectors=api.load("glove-wiki-gigaword-100")withopen("frequencies.json","r")asf:word_frequencies=json.load(f)embedding=Embedding.load_from_keyedvectors(word_vectors,frequencies=word_frequencies)neigh_dist,neigh_ind=embedding.compute_neighborhoods()EmbeddingComparisonTheEmbeddingComparisonclass is meant to compare twoEmbeddingobjects :fromembcompareimportEmbeddingComparison,load_embeddingemb1=load_embedding("first_emb.bin",embedding_format="fasttext",frequencies_path="freqs.pkl")emb2=load_embedding("second_emb.bin",embedding_format="word2vec",frequencies_path="freqs.pkl")comparison=EmbeddingComparison({"emb1":emb1,"emb2":emb2},n_neighbors=25)comparison.neighborhoods_similarities["word"]# 0.867JSON reportsEmbeddingReportTheEmbeddingReportclass is used to generate small report about an embedding :fromembcompareimportEmbeddingReport,load_embeddingemb1=load_embedding("first_emb.bin",embedding_format="fasttext",frequencies_path="freqs.pkl")report=EmbeddingReport(emb1)report.to_dict()# {#   "vector_size": 300,#   "mean_frequency": 0.00012,#   "mean_distance_neighbors": 0.023,#   ...# }EmbeddingComparisonReportTheEmbeddingComparisonReportclass is used to generate small comparison report from two embedding :fromembcompareimportEmbeddingComparison,EmbeddingComparisonReport,load_embeddingemb1=load_embedding("first_emb.bin",embedding_format="fasttext",frequencies_path="freqs.pkl")emb2=load_embedding("second_emb.bin",embedding_format="word2vec",frequencies_path="freqs.pkl")comparison=EmbeddingComparison({"emb1":emb1,"emb2":emb2})report=EmbeddingComparisonReport(comparison)report.to_dict()# {#   "embeddings" : [#     {#       "vector_size": 300,#       "mean_frequency": 0.00012,#       "mean_distance_neighbors": 0.023,#       ...#     },#     ...#   ],#   "neighborhoods_similarities_median": 0.012,#   ...# }📊 Create your custom streamlit appThe GUI is built withstreamlit. We tried to modularized the app so you can
more easily reuse some features for your custom streamlit app :# embcompare/gui/app.pyfromembcompare.gui.featuresimport(display_custom_elements_comparison,display_elements_comparison,display_embeddings_config,display_frequencies_comparison,display_neighborhoods_similarities,display_numbers_of_elements,display_parameters_selection,display_spaces_comparison,display_statistics_comparison,)fromembcompare.gui.helpersimportcreate_comparisondefmain():"""Streamlit app for embeddings comparison"""config_embeddings=config[CONFIG_EMBEDDINGS](tab_infos,tab_stats,tab_spaces,tab_neighbors,tab_compare,tab_compare_custom,tab_frequencies,)=st.tabs(["Infos","Statistics","Spaces","Similarities","Elements","Search elements","Frequencies",])# Embedding selection (inside the sidebar)withst.sidebar:parameters=display_parameters_selection(config_embeddings)# Display informations about embeddingswithtab_infos:display_embeddings_config(config_embeddings,parameters.emb1_id,parameters.emb2_id)comparison=create_comparison(config_embeddings,emb1_id=parameters.emb1_id,emb2_id=parameters.emb2_id,n_neighbors=parameters.n_neighbors,max_emb_size=parameters.max_emb_size,min_frequency=parameters.min_frequency,)# Display number of element in both embedding and common elementswithtab_infos:display_numbers_of_elements(comparison)# Display statisticswithtab_stats:display_statistics_comparison(comparison)ifnotcomparison.common_keys:st.warning("The embeddings have no element in common")st.stop()# Comparison below are based on common elements comparisonwithtab_spaces:display_spaces_comparison(comparison)withtab_neighbors:display_neighborhoods_similarities(comparison)withtab_compare:display_elements_comparison(comparison)withtab_compare_custom:display_custom_elements_comparison(comparison)withtab_frequencies:display_frequencies_comparison(comparison)