Repository: graspologic-org/graspologic

Issue:
implement spectral vertex nomination
https://github.com/neurodata/primitives-interfaces/tree/master/jhu_primitives/sgvn


Discussion:
@hhelm10 can you add a link to the associated paper for this? 
https://arxiv.org/pdf/1905.01776.pdf
Id like to work on this!
Also interested!
Also interested!
@hhelm10 think you could give us a quick rundown of spectral (single graph) VN? My original thought was a simple interface that eats a graph or embedding and does nearest neighbor queries... but I see now that there are more complicated schemes even before the ILP stuff. 
sure -- 

will give a short and sweet description of a set of methods.. 

i) start with an n by n graph G and a vertex of interest v*;. 
ii) pick your favorite embedding method to get n points in R^{d} (each d-dimensional vector is to be interpreted as a representation of a vertex in G);
iii) then, depending on the embedding method you use, pick a distance / dissimilarity on R^{d} (for instance, if i suspect my graph is generated from an SBM and i embed the graph using ASE / LSE i may choose either Euclidean distance for simplicity or Mahalanobis distance to exploit the assumed structure of the point in R^{d});
iv) calculate the distances between the vector corresponding to v* and vectors corresponding to the other n - 1 vertices;
v) return the closest k vertices to v* says your selected distance;

i can see an interface where you initialize the object with an embedding method and a distance, the "fit" method taking G, and the "predict" method taking v* and (optionally) k. 

does that make sense?
Thanks @hhelm10 , that's very helpful. I've been reading some papers on this, and have been a bit confused by the wide variety of similar, but not identical, methodologies. 
yea -- a lot of the methodologies that people come up w are pretty application / domain specific and so there are small tweaks here and there that try to optimize performance for the particular class of graph that they have access to (i.e. use LSE if graph is particularly sparse, somehow incorporate node / edge attributes, etc.). but for graphs larger than 50 nodes the typical procedure is to embed and apply / learn a distance to then rank the rest of the nodes.

a small aside: some of our recent work tries to avoid having to select an embedding method / distance by combining a bunch of them using supervisory information.  
Here would be my first draft of a VertexRanker class. Note that embedding_method is an instantiated embedding method and that pairwise_distance is not defined:

```
class VertexRanker:
    def __init__(embedding_method=ASE,
                      distance='Euclidean',
                      distance_kwargs = None):
        self.embedding_method = embedding_method
        self.distance = distance
        self.distance_kwargs = distance_kwargs

    def fit(G):
        X_hat = self.embedding_method.fit_transform(G)
        self.dist_matrix = pairwise_distance(X_hat, distance=self.distance, distance_kwargs=self.distance_kwargs)
        self.fitted = True

    def predict(index, k=1):
        return np.argsort(self.dist_matrix[index])[1:k+1]
```
@bdpedigo then to include something like the ILP you could let `embedding_method, distance, distance_kwargs` be a dictionaries and just make the fit function a little more complicated
DoD for this issue: 

**Minimum Features**

- A new vertex nomination class, with a spectral VN subclass that can take an adj. matrix and a list of seed vectors and associated labels, and can return Nomination Lists of Vertices for each class of interest ordered by probability.
- Should have options for different embedding methods, and use existing implementations.
- should have options for different clustering methods, particularly kmeans and gmm. 
- should run very quickly, even for graphs with millions of vertices. 
- All new classes and methods will be properly documented.
- A short tutorial notebook will be made to demonstrate functionality.

**Checks**

- Should preform very well on synthetic SBM data,  ~100% accurate (for the first K nominees, where k is # vertices in block) even without many seed vertices.
- Should be able to run on the enron dataset (http://research.cs.queensu.ca/home/skill/siamworkshop.html) and be able to approximately reproduce the performance result shown in  _Fishkind, et. al., Vertex Nomination for Membership Prediction, figure 3_. 
- Should be able to successfully nominate vertices with matching cell type labels from the drosophila dataset included with graspologic. 

**Ideal Features**
_Planning to do pretty much all of these, but goes beyond the basic functionality_
- should be extended into the multigraph case, e.g. should be able to give seed vertices in graph A, and nominate vertices from related graph B. 
- Should have the option to choose different distance metrics to be used in clustering
- Should be able to use semi-supervised kmeans and gmm for clustering (in standard implementation, seed vertices are essentially wasted except for deciding cluster label)
- Should have a tutorial notebook going through a more detailed, real world use case.
- Should be able to optionally infer K (number of blocks)

> can take an adj. matrix

maybe also an embedding

> for each class of interest ordered by probability.

You may have already read up on VN more than me, which is great - I get the "class nomination" idea but I believe there should also be a completely unsupervised notion (see @hhelm10's example implementation, which has no notion of class) 

> should run very quickly, even for graphs with millions of vertices.

Would be awesome, but first goal is to just get something that works. also "very quickly" is vague

> Should have the option to choose different distance metrics to be used in clustering

For something like GMM and maybe even k-means just note that this is a non-trivial problem.

> Should be able to optionally infer K (number of blocks)

also non-trivial problem

other notes: 
- would like the interface to match that of other nomination classes/functions as much as makes sense. 
- i think a class interface will make the most sense

Resolving patch:
diff --git a/graspologic/embed/__init__.py b/graspologic/embed/__init__.py
--- a/graspologic/embed/__init__.py
+++ b/graspologic/embed/__init__.py
@@ -8,8 +8,10 @@
 from .n2v import node2vec_embed
 from .omni import OmnibusEmbed
 from .svd import select_dimension, selectSVD
+from .base import BaseSpectralEmbed
 from .mug2vec import mug2vec
 
+
 __all__ = [
     "ClassicalMDS",
     "OmnibusEmbed",
@@ -19,4 +21,5 @@
     "node2vec_embed",
     "select_dimension",
     "selectSVD",
+    "BaseSpectralEmbed",
 ]
diff --git a/graspologic/nominate/__init__.py b/graspologic/nominate/__init__.py
--- a/graspologic/nominate/__init__.py
+++ b/graspologic/nominate/__init__.py
@@ -1,6 +1,10 @@
 # Copyright (c) Microsoft Corporation and contributors.
 # Licensed under the MIT License.
 
+from .spectralVN import SpectralVertexNomination
 from .VNviaSGM import VNviaSGM
 
-__all__ = ["VNviaSGM"]
+__all__ = [
+    "SpectralVertexNomination",
+    "VNviaSGM",
+]
diff --git a/graspologic/nominate/spectralVN.py b/graspologic/nominate/spectralVN.py
new file mode 100644
--- /dev/null
+++ b/graspologic/nominate/spectralVN.py
@@ -0,0 +1,259 @@
+# Copyright (c) Microsoft Corporation and contributors.
+# Licensed under the MIT License.
+
+from typing import Union, Tuple
+from ..embed import BaseSpectralEmbed
+from ..embed import AdjacencySpectralEmbed, LaplacianSpectralEmbed
+import numpy as np
+from sklearn.neighbors import NearestNeighbors
+from sklearn.base import BaseEstimator
+
+
+class SpectralVertexNomination(BaseEstimator):
+    """
+    Class for spectral vertex nomination on a single graph.
+
+    Given a graph :math:`G=(V,E)` and a subset of :math:`V` called :math:`S`
+    (the "seed"), Single Graph Vertex Nomination is the problem of ranking all :math:`V`
+    in order of relation to members of :math:`S`. Spectral Vertex Nomination solves
+    this problem by embedding :math:`G` into a low dimensional euclidean space
+    (:ref:`tutorials <embed_tutorials>`), and then generating a nomination list by some
+    distance based algorithm. In the simple unattributed case, for each seed vertex
+    :math:`u`, the other vertices are ranked in order of euclidean distance from
+    :math:`u`.
+
+    Parameters
+    ----------
+    input_graph: bool, default = True
+        Flag whether to expect two full graphs, or the embeddings.
+
+        - True
+            .fit and .fit_predict() expect graphs as adjacency matrix, provided as
+            ndarray of shape (n, n). They will be embedded using the specified
+            ``embedder``.
+        - False
+            .fit() and .fit_predict() expect an embedding of the graph, i.e. a ndarray
+            of size (n, d).
+    embedder: str or BaseEmbed, default = 'ASE'
+        May provide either a embed object or a string indicating which embedding method
+        to use, which may be either:
+        "ASE" for :py:class:`~graspologic.embed.AdjacencySpectralEmbed` or
+        "LSE" for :py:class:`~graspologic.embed.LaplacianSpectralEmbed`.
+    n_neighbors: int, default=None
+        The number of vertices to nominate for each seed.
+    metric : str, default = 'euclidean'
+        Distance metric to use when finding the nearest neighbors, all sklearn metrics
+        available.
+    metric_params : dict, default = None
+        Arguments for the sklearn `DistanceMetric` specified via `metric` parameter.
+
+    Attributes
+    ----------
+    nearest_neighbors_ : sklearn.neighbors.NearestNeighbors
+        A fit sklearn `NearestNeighbors` classifier used to find closest vertices to
+        each seed.
+
+    References
+    ----------
+    .. [1] Fishkind, D. E.; Lyzinski, V.; Pao, H.; Chen, L.; Priebe, C. E. Vertex
+        nomination schemes for membership prediction. Ann. Appl. Stat. 9 2015.
+        https://projecteuclid.org/euclid.aoas/1446488749
+
+    .. [2] Jordan Yoder, Li Chen, Henry Pao, Eric Bridgeford, Keith Levin,
+        Donniell E. Fishkind, Carey Priebe, Vince Lyzinski, Vertex nomination: The
+        canonical sampling and the extended spectral nomination schemes, Computational
+        Statistics & Data Analysis, Volume 145, 2020.
+        http://www.sciencedirect.com/science/article/pii/S0167947320300074
+
+    """
+
+    def __init__(
+        self,
+        input_graph: bool = True,
+        embedder: Union[str, BaseSpectralEmbed] = "ase",
+        n_neighbors=None,
+        metric: str = "euclidean",
+        metric_params: dict = None,
+    ):
+        super().__init__()
+        self.embedder = embedder
+        self.input_graph = input_graph
+        self.n_neighbors = n_neighbors
+        self.metric = metric
+        self.metric_params = metric_params
+        self._check_params()
+
+    def _check_x(self, X: np.ndarray):
+        # check X
+        if not isinstance(X, np.ndarray):
+            raise TypeError("X must be of type np.ndarray.")
+        elif not np.issubdtype(X.dtype, np.number):
+            raise TypeError("Adjacency matrix or embedding should have numeric type")
+        elif np.ndim(X) != 2:
+            raise IndexError("Adjacency matrix or embedding must have dim 2")
+        elif not self.input_graph:
+            # embedding was provided
+            if X.shape[1] > X.shape[0]:
+                raise IndexError("Dim 1 of an embedding should be smaller than dim 0.")
+            if not np.issubdtype(X.dtype, np.float):
+                raise TypeError("Embedding should have type float")
+        elif not np.issubdtype(X.dtype, np.int):
+            raise TypeError("Adjacency matrix should have type int")
+        elif X.shape[0] != X.shape[1]:
+            raise IndexError("Adjacency Matrix should be square.")
+
+    def _check_y(self, y: np.ndarray):
+        # check y
+        if not isinstance(y, np.ndarray):
+            raise TypeError("y must be of type np.ndarray")
+        elif not np.issubdtype(y.dtype, np.integer):
+            raise TypeError("y must have dtype int")
+        elif np.ndim(y) > 2 or (y.ndim == 2 and y.shape[1] > 1):
+            raise IndexError("y must have shape (n) or (n, 1).")
+        elif y.shape[0] > self.embedding_.shape[0]:
+            raise ValueError(
+                "the number of seeds must be less than the number of vertices"
+            )
+
+    def _check_params(self):
+
+        if self.n_neighbors is not None and type(self.n_neighbors) is not int:
+            raise TypeError("k must be an integer")
+        elif self.n_neighbors is not None and self.n_neighbors <= 0:
+            raise ValueError("k must be greater than 0")
+        if type(self.metric) is not str:
+            raise TypeError("metric must be a string")
+        if self.metric_params is not None and type(self.metric_params) is not dict:
+            raise TypeError("metric_params must be a dictionary")
+        if type(self.input_graph) is not bool:
+            raise TypeError("input_graph_ must be of type bool.")
+        if self.input_graph:
+            if not isinstance(self.embedder, BaseSpectralEmbed) and not isinstance(
+                self.embedder, str
+            ):
+                raise TypeError(
+                    "embedder must be either of type str or BaseSpectralEmbed"
+                )
+
+    def _embed(self, X: np.ndarray):
+        # Embed graph if embedding not provided
+        if self.input_graph:
+            if isinstance(self.embedder, BaseSpectralEmbed):
+                embedder = self.embedder
+            elif not isinstance(self.embedder, str):
+                raise TypeError("embedder must be type str or BaseSpectralEmbed")
+            elif self.embedder.lower() == "ase":
+                embedder = AdjacencySpectralEmbed()
+            elif self.embedder.lower() == "lse":
+                embedder = LaplacianSpectralEmbed()
+            else:
+                raise ValueError(
+                    "Requested embedding method does not exist, if str is passed must "
+                    "be either 'ASE' or 'LSE'."
+                )
+            self.embedding_ = embedder.fit_transform(X)
+        else:
+            self.embedding_ = X
+
+    def fit(
+        self,
+        X: np.ndarray,
+        y=None,
+    ):
+        """
+        Constructs the embedding if not provided, then calculates the pairwise distance
+        from each seed to each vertex in graph.
+
+        Parameters
+        ----------
+        X : np.ndarray
+            - If `input_graph` is True
+                Expects a graph as an adjacency matrix, i.e. an ndarray of shape (n, n).
+                Will be embedded using the specified embedder.
+            - If `input_graph` is False
+                Expects an embedding of the graph, i.e. a ndarray of size (n, d).
+        y : NoneType
+            Included by sklearn convention.
+
+        Returns
+        -------
+        self : object
+            Returns an instance of self.
+        """
+        # ensure y has correct shape. If unattributed (1d)
+        # add unique attribute to each seed vertex.
+        self._check_x(X)
+        self._embed(X)
+        if self.n_neighbors is None:
+            self.n_neighbors = X.shape[0]
+        self.nearest_neighbors_ = NearestNeighbors(
+            n_neighbors=self.n_neighbors,
+            metric=self.metric,
+            metric_params=self.metric_params,
+        )
+        self.nearest_neighbors_.fit(self.embedding_)
+        return self
+
+    def predict(self, y: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
+        """
+        Nominates vertices for each seed vertex. Methodology is distance based ranking.
+
+
+        Parameters
+        ----------
+        y : np.ndarray
+          The indices of the seed vertices. Should be a dim 1 array with length less
+          than :math:`|V|`.
+
+        Returns
+        -------
+        Nomination List : np.ndarray
+                        Shape is ``(number_vertices, number_vertices_in_seed)`` . Each
+                        column is a seed vertex, and the rows of each
+                        column are a list of vertex indexes from the original adjacency
+                        matrix in order degree of match.
+        Distance Matrix : np.ndarray
+                        The matrix of distances associated with each element of the
+                        nomination list.
+        """
+        self._check_y(y)
+        y = y.reshape(-1)
+        y_vec = self.embedding_[y.astype(np.int)]
+        if not hasattr(self, "nearest_neighbors_"):
+            raise ValueError("Fit must be called before predict.")
+        distance_matrix, nomination_list = self.nearest_neighbors_.kneighbors(y_vec)
+        # transpose for consistency with literature
+        return nomination_list.T.astype(np.int), distance_matrix.T
+
+    def fit_predict(
+        self,
+        X: np.ndarray,
+        y: np.ndarray,
+    ) -> Tuple[np.ndarray, np.ndarray]:
+        """
+        Calls this class' fit and then predict methods.
+
+        Parameters
+        ----------
+        X : np.ndarray
+            - If `input_graph` is True
+                Expects a graph as an adjacency matrix, i.e. an ndarray of shape (n, n).
+                Will be embedded using the specified embedder.
+            - If `input_graph` is False
+                Expects an embedding of the graph, i.e. a ndarray of size (n, d).
+        y : np.ndarray.
+            List of unattributed seed vertex indices.
+
+        Returns
+        -------
+        Nomination List : np.ndarray
+                        Shape is ``(number_vertices, number_vertices_in_seed)`` . Each
+                        column is a seed vertex, and the rows of each
+                        column are a list of vertex indexes from the original adjacency
+                        matrix in order degree of match.
+        Distance Matrix : np.ndarray
+                        The matrix of distances associated with each element of the
+                        nomination list.
+        """
+        self.fit(X)
+        return self.predict(y)


Test patch:
diff --git a/tests/test_spectral_nomination.py b/tests/test_spectral_nomination.py
new file mode 100644
--- /dev/null
+++ b/tests/test_spectral_nomination.py
@@ -0,0 +1,168 @@
+import pytest
+import numpy as np
+from graspologic.embed.ase import AdjacencySpectralEmbed
+from graspologic.simulations.simulations import sbm
+from graspologic.nominate import SpectralVertexNomination
+
+# global constants for tests
+n_verts = 50
+p = np.array([[0.7, 0.25, 0.2], [0.25, 0.8, 0.3], [0.2, 0.3, 0.85]])
+labels = np.array([0] * n_verts + [1] * n_verts + [2] * n_verts)
+adj = np.array(sbm(3 * [n_verts], p), dtype=np.int)
+embeder = AdjacencySpectralEmbed()
+pre_embeded = embeder.fit_transform(adj)
+
+
+def _nominate(X, seed, nominator=None, k=None):
+    if nominator is None:
+        nominator = SpectralVertexNomination(n_neighbors=k)
+    nominator.fit(X)
+    n_verts = X.shape[0]
+    nom_list, dists = nominator.predict(seed)
+    assert nom_list.shape == (n_verts, seed.shape[0])
+    assert dists.shape == (n_verts, seed.shape[0])
+    return nom_list
+
+
+def _test_seed_input_dimensions():
+    with pytest.raises(IndexError):
+        _nominate(adj, np.zeros((5, 5, 5), dtype=np.int))
+
+
+def _test_seed_shape():
+    with pytest.raises(IndexError):
+        _nominate(adj, np.zeros((1, 50), dtype=np.int))
+
+
+def _test_seed_input_array_dtype():
+    with pytest.raises(TypeError):
+        _nominate(adj, np.random.random((10, 2)))
+
+
+def _test_seed_input_type():
+    with pytest.raises(TypeError):
+        _nominate(adj, [0] * 10)
+
+
+def _test_X_input_type():
+    with pytest.raises(TypeError):
+        _nominate([[0] * 10] * 10, np.zeros(3, dtype=np.int))
+
+
+def _test_X_array_dtype():
+    with pytest.raises(TypeError):
+        _nominate(np.random.random((10, 10)), np.zeros(3, dtype=np.int))
+
+
+def _test_X_input_dimensions():
+    with pytest.raises(IndexError):
+        _nominate(np.zeros((5, 5, 5), dtype=np.int), np.zeros(3, dtype=np.int))
+
+
+def _test_embedding_dimensions():
+    # embedding should have less cols than rows.
+    svn = SpectralVertexNomination(input_graph=False)
+    with pytest.raises(IndexError):
+        _nominate(
+            np.zeros((10, 20), dtype=np.int), np.zeros(3, dtype=np.int), nominator=svn
+        )
+
+
+def _test_adjacency_shape():
+    # adj matrix should be square
+    with pytest.raises(IndexError):
+        _nominate(np.zeros((3, 4), dtype=np.int), np.zeros(3, dtype=np.int))
+
+
+def _test_input_graph_bool_type():
+    # input graph param has wrong type
+    with pytest.raises(TypeError):
+        svn = SpectralVertexNomination(input_graph=4)
+
+
+def _test_k_type():
+    # k of wrong type
+    with pytest.raises(TypeError):
+        _nominate(adj, np.zeros(3, dtype=np.int), k="hello world")
+
+
+def _test_k_value():
+    # k should be > 0
+    with pytest.raises(ValueError):
+        _nominate(adj, np.zeros(3, dtype=np.int), k=0)
+
+
+def _test_embedder_type():
+    # embedder must be BaseSpectralEmbed or str
+    with pytest.raises(TypeError):
+        svn = SpectralVertexNomination(embedder=45)
+
+
+def _test_embedder_value():
+    with pytest.raises(ValueError):
+        svn = SpectralVertexNomination(embedder="hi")
+        _nominate(adj, np.zeros(3, dtype=int), nominator=svn)
+
+
+class TestSpectralVertexNominatorOutputs:
+    def test_seed_inputs(self):
+        _test_seed_input_dimensions()
+        _test_seed_shape()
+        _test_seed_input_array_dtype()
+        _test_seed_input_type()
+
+    def test_X_inputs(self):
+        _test_X_array_dtype()
+        _test_X_input_dimensions()
+        _test_X_input_type()
+        _test_embedding_dimensions()
+        _test_adjacency_shape()
+
+    def _test_k(self):
+        _test_k_value()
+        _test_k_type()
+
+    def test_constructor_inputs(self):
+        _test_embedder_value()
+
+    def test_constructor_inputs1(self):
+        _test_embedder_type()
+
+    def test_constructor_inputs2(self):
+        _test_input_graph_bool_type()
+
+    @pytest.mark.parametrize(
+        "nominator",
+        [
+            SpectralVertexNomination(embedder="ASE"),
+            SpectralVertexNomination(embedder="LSE"),
+            SpectralVertexNomination(embedder=embeder),
+        ],
+    )
+    @pytest.mark.parametrize(
+        "seed",
+        [
+            np.array([8]),
+            np.array([2, 6, 9, 15, 25]),
+            np.arange(n_verts - 1, dtype=np.int),
+        ],
+    )
+    def test_basic_unattributed(self, nominator, seed):
+        """
+        Runs two attributed seeds and two unattributed seeds with each nominator.
+        Ensures all options work. Should be fast. Nested parametrization tests all
+        combinations of listed parameters.
+        """
+        _nominate(adj, seed, nominator)
+
+    @pytest.mark.parametrize(
+        "seed",
+        [
+            np.array([8]),
+            np.array([2, 6, 9, 15, 25]),
+            np.arange(n_verts - 1, dtype=np.int),
+        ],
+    )
+    def test_pre_embedded(self, seed):
+        svn = SpectralVertexNomination(input_graph=False)
+        _nominate(pre_embeded, seed, nominator=svn)