Package: chem_wasserstein
chem_wassersteinA high performance mapping class to constructElMD distance matricesfrom large datasets of ionic compositions, suitable for single node usage on HPC systems. This includes helper methods to directly embed these datasets as maps of chemical space, as well as sorting lists of compositions, and exporting kernel matrices.This Fork:warning: For the original ElM2D repository, seehttps://github.com/lrcfmd/ElM2D. :warning:This is a refactored version which incorporates fast distance matrix computations for the modified Pettifor scale representation, and is in the process of being integrated into ElMD and ElM2D. The documentation as follows has changes relative to the original documentation. Additionally, this is packaged on PyPI and Anaconda, but under a different name:chem_wasserstein. A $10000 \times 10000$ pairwise distance matrix can be calculated on the order of 10 seconds (CPU). The distance calculations are also GPU compatible. If running on Colab, be sure to use GPU as the CPU version has some Colab-specific issues with Numba.InstallationRecommended installation throughcondawith python 3.8.conda install -c sgbaird chem_wassersteinorpip install chem_wassersteinFor the background theory, please read the paper"The Earth Mover’s Distance as a Metric for the Space of Inorganic Compositions"Examples125,000 compositions from the inorganic crystal structure database embedded with PCA, plotted withdatashader:For more interactive examples please seewww.elmd.io/plotsUsageComputing Distance MatricesThe computed distance matrix is accessible through thedmattribute and can be saved and loaded as a csv file.fromchem_wasserstein.ElM2D_importElM2Dmapper=ElM2D()mapper.fit(df["formula"])print(mapper.dm)mapper.export_dm("ComputedMatrix.csv")This distance matrix can be used as a lookup table for distances between compositions given their numeric indices (distance = mapper.dm[i][j]) or used as a kernel matrix for embedding, regression, and classification tasks directly.SortingTo sort a list of compositions into an ordering of chemical similaritymapper.fit(df["formula"])sorted_indices=mapper.sort()sorted_comps=mapper.sorted_formulasEmbeddingEmbeddings can be constructed through either theUMAPor PCA methods of dimensionality reduction. The most recently embedded points are accessible via theembeddingproperty. Higher dimensional embeddings can be created with then_componentsparameter.mapper=ElM2D()mapper.fit(df["formula"])embedding=mapper.transform()...# For new dataembedding=mapper.fit_transform(df["formula"])embedding=mapper.fit_transform(df["formula"],how="PCA",n_components=7)Embeddings may also be directed towards a particular chemical property in a pandas DataFrame, to bring known patterns into focus.embedding=mapper.fit_transform(df["formula"],df["property_of_interest"])By default, themodified Pettifor scaleis used as the method of atomic similarity, this is changed through themetricattribute.mapper=ElM2D(metric="atomic")embedding=mapper.fit_transform(df["formula"])These embeddings may be visualized within a jupyter notebook, or exported to HTML to view full page in the web browser.mapper.fit_transform(df["formula"])# Returns a figure for viewing in notebooksmapper.plot()# Returns a figure and saves as ElM2D_Plot_UMAP.htmlmapper.plot("ElM2D_Plot_UMAP.html")# Returns and saves figure, with colouring based on property from a pandas Seriesmapper.plot(fp="ElM2D_Plot_UMAP.html",color=df["chemical_property"])# Plotting also works in 3Dmapper.fit_transform(df["formula"],n_components=3)mapper.plot(color=df["chemical_property"])SavingSmaller datasets can be saved directly with thesave(filepath.pk)/load(filepath.pk)methods directly. This is limited to files of size 3GB (the python binary file size limit).Larger datasets will require importing/exporting the distance matrix and embeddings (export_embedding(filepath.csv)/import_embedding(filepath.csv)separately as csv files if you require this processed data in future work.mapper.fit(small_df["formula"])mapper.save("small_df_mapper.pk")...mapper=ElM2D()mapper.load("small_df_mapper.pk")...mapper.fit(large_df["formula"])mapper.export_dm("large_df_dm.csv")mapper.export_embedding("large_df_emb_UMAP.csv")...mapper=ElM2D()mapper.import_dm("large_df_dm.csv")mapper.import_embedding("large_df_emb_UMAP.csv")mapper.formula_list=df["formula"]Cross ValidationPerform a K-Folds splitting of the dataset into subsets, to build up training
and testing datasets.cvs=mapper.cross_validate()fori,(X_train,X_test)inenumerate(cvs):sub_mapper=ElM2D()sub_mapper.fit(X_train)sub_mapper.save(f"train_elm2d_{i}.pk")sub_mapper.fit(X_test)sub_mapper.save(f"test_elm2d_{i}.pk")...fromsklearn.metricsimportmean_average_errorasmaecvs=mapper.cross_validate(y=df["target"])forX_train,X_test,y_train,y_testincvs:clf.fit(X_train,y_train)y_pred=clf.predict(X_test)errors.append(mae(y_pred,y_test))print(np.mean(errors))Available MetricsYou may use either discrete scales or machine learnt representations for each element. Choose these via themetricparameter.Linear:mendeleevpettiatomicmod_pettiChemically Derived:oliynykoliynyk_scjarvisjarvis_scmagpiemagpie_scMachine Learnt:cgcnnelemnetmat2vecmatscholarmegnet16Random Numbers:random_200Custom Distance MatrixprecomputedBulk featurizing can be performed withfeaturize.mapper=ElM2D(metric="oliynyk_sc")X=mapper.featurize(df["formula"])CitingIf you would like to cite this code in your work, please use the following reference@article{doi:10.1021/acs.chemmater.0c03381,
author = {Hargreaves, Cameron J. and Dyer, Matthew S. and Gaultois, Michael W. and Kurlin, Vitaliy A. and Rosseinsky, Matthew J.},
title = {The Earth Mover’s Distance as a Metric for the Space of Inorganic Compositions},
journal = {Chemistry of Materials},
volume = {32},
number = {24},
pages = {10610-10620},
year = {2020},
doi = {10.1021/acs.chemmater.0c03381},
URL = { 
        https://doi.org/10.1021/acs.chemmater.0c03381
},
eprint = { 
        https://doi.org/10.1021/acs.chemmater.0c03381
}
}