Repository: graspologic-org/graspologic

Issue:
implement mixed membership SBM sampling



Discussion:
@jovo do we want this? I notice it's in [COSIE paper](https://arxiv.org/pdf/1906.10026.pdf) and I've seen it in the literature enough to think maybe we do (not just as a special case of RDPG) 
i only care if we are going to use it to answer a question

On Fri, Sep 11, 2020 at 7:08 PM Benjamin Pedigo <notifications@github.com>
wrote:

>
> *      External Email - Use Caution      *
>
>
>
> @jovo
> <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjovo&data=02%7C01%7Cjovo%40jhu.edu%7C5f2b7f9e34e94120104e08d856a7a8c8%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637354625823354960&sdata=tnT0cKq%2BxkDfBc%2F5%2FEMc7yf%2BHMAlxfyuqCnub8wwDJY%3D&reserved=0>
> do we want this? I notice it's in COSIE paper
> <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fpdf%2F1906.10026.pdf&data=02%7C01%7Cjovo%40jhu.edu%7C5f2b7f9e34e94120104e08d856a7a8c8%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637354625823364955&sdata=YlmmvyZsv05PZXM8X2ZFHNo10HR9dRQiapONgPdkV4M%3D&reserved=0>
> and I've seen it in the literature enough to think maybe we do (not just as
> a special case of RDPG)
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fneurodata%2Fgraspy%2Fissues%2F440%23issuecomment-691348396&data=02%7C01%7Cjovo%40jhu.edu%7C5f2b7f9e34e94120104e08d856a7a8c8%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637354625823364955&sdata=8%2FY4U0Mp%2FrbHIu1WftG9%2BK8UmijdIsv9VnjAx2hbV6Q%3D&reserved=0>,
> or unsubscribe
> <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAAKG4SI3GEB624VXJGRW53SFKUYNANCNFSM4RIQZLTA&data=02%7C01%7Cjovo%40jhu.edu%7C5f2b7f9e34e94120104e08d856a7a8c8%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637354625823374946&sdata=GjqAy90HPpF2aDW%2B0nFxBt9R%2BTkLfF6CLGpHFYUnJ18%3D&reserved=0>
> .
>


-- 
With gratitude,

Joshua T Vogelstein, PhD
neurodata.io | BME@JHU <https://www.bme.jhu.edu/> | dA/dt > 0 > dJ/dt
<https://twitter.com/neuro_data/status/1279067902658916352>
where A = appreciating, J = judging, and t = time.
Think I can do better? Tell me how <https://forms.gle/iEad8byD89eTPdYx6>!

used in anomaly detection and COSIE papers. so i feel like it'd be nice to have, at a bare minimum to be able to replicate/build off of those results

@j1c did you have to implement any MMSBM stuff for those? 
@bdpedigo I will take this one!

DoD: 

- Add mixed membership stochastic block model(MMsbm) sampling function to simulations.py:
    - Must accept as input the number of vertices of the network, the block connectivity matrix B and the parameter alpha for 
       sampling of mixed membership vectors from Dirichlet distribution
    - Must output resulting adjacency matrix
- Written documentation, listing all inputs and outputs
- Written tutorial that shows simple functionality and difference with SBM

Tests: 

- Replicate Figure 6 of the [MMsbm paper](https://jmlr.csail.mit.edu/papers/volume9/airoldi08a/airoldi08a.pdf)  
- Check if error messages are displayed for all possible types of mismatches between input provided by user and input required by the function

P.S. This is only an initial draft, I am open to suggestions. I will populate the test section and provide a more detailed description of inputs and outputs this week. I am open to suggestions in regards of possible additional functionalities.



Resolving patch:
diff --git a/graspologic/simulations/__init__.py b/graspologic/simulations/__init__.py
--- a/graspologic/simulations/__init__.py
+++ b/graspologic/simulations/__init__.py
@@ -1,7 +1,7 @@
 # Copyright (c) Microsoft Corporation and contributors.
 # Licensed under the MIT License.
 
-from .simulations import sample_edges, er_np, er_nm, sbm, rdpg, p_from_latent
+from .simulations import sample_edges, er_np, er_nm, sbm, rdpg, p_from_latent, mmsbm
 from .simulations_corr import sample_edges_corr, er_corr, sbm_corr
 from .rdpg_corr import rdpg_corr
 
@@ -16,4 +16,5 @@
     "er_corr",
     "sbm_corr",
     "rdpg_corr",
+    "mmsbm",
 ]
diff --git a/graspologic/simulations/simulations.py b/graspologic/simulations/simulations.py
--- a/graspologic/simulations/simulations.py
+++ b/graspologic/simulations/simulations.py
@@ -4,6 +4,7 @@
 import numpy as np
 
 from ..utils import symmetrize, cartprod
+from sklearn.utils import check_array, check_scalar
 import warnings
 
 
@@ -795,3 +796,184 @@ def p_from_latent(X, Y=None, rescale=False, loops=True):
         P[P < 0] = 0
         P[P > 1] = 1
     return P
+
+
+def mmsbm(
+    n,
+    p,
+    alpha=None,
+    rng=None,
+    directed=False,
+    loops=False,
+    return_labels=False,
+):
+    r"""
+    Samples a graph from Mixed Membership Stochastic Block Model (MMSBM).
+
+    MMSBM produces a graph given the specified block connectivity matrix B,
+    which indicates the probability of connection between nodes based upon
+    their community membership.
+    Each node is assigned a membership vector drawn from Dirichlet distribution
+    with parameter :math:`\vec{\alpha}`. The entries of this vector indicate the
+    probabilities for that node of pertaining to each community when interacting with
+    another node. Each node's membership is determined according to those probabilities.
+    Finally, interactions are sampled according to the assigned memberships.
+
+    Read more in the :ref:`tutorials <simulations_tutorials>`
+
+    Parameters
+    ----------
+    n: int
+        Number of vertices of the graph.
+
+    p: array-like, shape (n_communities, n_communities)
+        Probability of an edge between each of the communities, where ``p[i, j]``
+        indicates the probability of a connection between edges in communities
+        :math:`(i, j)`.
+        0 < ``p[i, j]`` < 1 for all :math:`i, j`.
+
+    alpha: array-like, shape (n_communities,)
+        Parameter alpha of the Dirichlet distribution used
+        to sample the mixed-membership vectors for each node.
+        ``alpha[i]`` > 0 for all :math:`i`.
+
+    rng: numpy.random.Generator, optional (default = None)
+        :class:`numpy.random.Generator` object to sample from distributions.
+        If None, the random number generator is the Generator object constructed
+        by ``np.random.default_rng()``.
+
+    directed: boolean, optional (default=False)
+        If False, output adjacency matrix will be symmetric. Otherwise, output adjacency
+        matrix will be asymmetric.
+
+    loops: boolean, optional (default=False)
+        If False, no edges will be sampled in the diagonal. Otherwise, edges
+        are sampled in the diagonal.
+
+    return_labels: boolean, optional (default=False)
+        If False, the only output is the adjacency matrix.
+        If True, output is a tuple. The first element of the tuple is the adjacency
+        matrix. The second element is a matrix in which the :math:`(i^{th}, j^{th})`
+        entry indicates the membership assigned to node i when interacting with node j.
+        Community 1 is labeled with a 0, community 2 with 1, etc.
+        -1 indicates that no community was assigned for that interaction.
+
+    References
+    ----------
+    .. [1] Airoldi, Edoardo, et al. “Mixed Membership Stochastic Blockmodels.”
+       Journal of Machine Learning Research, vol. 9, 2008, pp. 1981–2014.
+
+    Returns
+    -------
+    A: ndarray, shape (n, n)
+        Sampled adjacency matrix
+    labels: ndarray, shape (n, n), optional
+        Array containing the membership assigned to each node when interacting with
+        another node.
+
+    Examples
+    --------
+    >>> rng = np.random.default_rng(1)
+    >>> np.random.seed(1)
+    >>> n = 6
+    >>> p = [[0.5, 0], [0, 1]]
+
+    To sample a binary MMSBM in which very likely all nodes pertain to community two:
+
+    >>> alpha = [0.05, 1000]
+    >>> mmsbm(n, p, alpha, rng = rng)
+    array([[0., 1., 1., 1., 1., 1.],
+           [1., 0., 1., 1., 1., 1.],
+           [1., 1., 0., 1., 1., 1.],
+           [1., 1., 1., 0., 1., 1.],
+           [1., 1., 1., 1., 0., 1.],
+           [1., 1., 1., 1., 1., 0.]])
+
+    To sample a binary MMSBM similar to 2-block SBM with connectivity matrix B:
+
+    >>> rng = np.random.default_rng(1)
+    >>> np.random.seed(1)
+    >>> alpha = [0.05, 0.05]
+    >>> mmsbm(n, p, alpha, rng = rng)
+    array([[0., 1., 0., 0., 0., 0.],
+           [1., 0., 0., 0., 0., 0.],
+           [0., 0., 0., 0., 0., 0.],
+           [0., 0., 0., 0., 1., 1.],
+           [0., 0., 0., 1., 0., 1.],
+           [0., 0., 0., 1., 1., 0.]])
+
+    """
+
+    check_scalar(x=n, name="n", target_type=int, min_val=1)
+
+    p = check_array(p, ensure_2d=True)
+    nx, ny = p.shape
+    if nx != ny:
+        msg = "p must be a square matrix, not {}".format(p.shape)
+        raise ValueError(msg)
+    if not np.issubdtype(p.dtype, np.number):
+        msg = "There are non-numeric elements in p"
+        raise ValueError(msg)
+    if np.any(p < 0) or np.any(p > 1):
+        msg = "Values in p must be in between 0 and 1."
+        raise ValueError(msg)
+
+    alpha = check_array(alpha, ensure_2d=False, ensure_min_features=1)
+    if not np.issubdtype(alpha.dtype, np.number):
+        msg = "There are non-numeric elements in alpha"
+        raise ValueError(msg)
+    if np.any(alpha <= 0):
+        msg = "Alpha entries must be > 0."
+        raise ValueError(msg)
+    if alpha.shape != (len(p),):
+        msg = "alpha must be a list or np.array of shape {c}, not {w}.".format(
+            c=(len(p),), w=alpha.shape
+        )
+        raise ValueError(msg)
+
+    if not isinstance(rng, (np.random._generator.Generator)):
+        msg = "rng must be <class 'numpy.random._generator.Generator'> not {}.".format(
+            type(rng)
+        )
+        raise TypeError(msg)
+    elif rng == None:
+        rng = np.random.default_rng()
+
+    if type(loops) is not bool:
+        raise TypeError("loops is not of type bool.")
+    if type(directed) is not bool:
+        raise TypeError("directed is not of type bool.")
+    if type(return_labels) is not bool:
+        raise TypeError("return_labels is not of type bool.")
+
+    if not directed:
+        if np.any(p != p.T):
+            raise ValueError("Specified undirected, but P is directed.")
+
+    # Naming convention follows paper listed in references.
+    mm_vectors = rng.dirichlet(alpha, n)
+
+    mm_vectors = np.array(sorted(mm_vectors, key=lambda x: np.argmax(x)))
+
+    # labels:(n,n) matrix with all membership indicators for initiators and receivers
+    # instead of storing the indicator vector, argmax is directly computed
+    # check docstrings for more info.
+    labels = np.apply_along_axis(
+        lambda p_vector: np.argmax(
+            rng.multinomial(n=1, pvals=p_vector, size=n), axis=1
+        ),
+        axis=1,
+        arr=mm_vectors,
+    )
+
+    P = p[(labels, labels.T)]
+
+    A = sample_edges(P, directed=directed, loops=loops)
+
+    if not loops:
+        np.fill_diagonal(labels, -1)
+
+    if return_labels:
+        return (A, labels)
+
+    return A


Test patch:
diff --git a/tests/test_sims.py b/tests/test_sims.py
--- a/tests/test_sims.py
+++ b/tests/test_sims.py
@@ -882,3 +882,179 @@ def test_kwarg_passing(self):
         g = rdpg(X, rescale=True, loops=False, directed=False)
         self.assertTrue(is_symmetric(g))
         self.assertTrue(is_loopless(g))
+
+
+class Test_MMSBM(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        # 120 vertex graph
+        cls.n = 120
+        # define connectivity matrix B, two blocks
+        cls.p = np.vstack(([0.3, 0.2], [0.2, 0.9]))
+        # define alpha parameter
+        cls.alpha = 0.05 * np.ones(len(cls.p))
+        # define seed
+        cls.seed = 12345
+        # define random number generator
+        cls.rng = np.random.default_rng(cls.seed)
+
+    def test_ER_np(self):
+        rng = np.random.default_rng(self.seed)
+        np.random.seed(self.seed)
+
+        p = [[1, 0.4], [0.4, 0.7]]
+
+        alpha = [1000, 0.05]
+
+        n = 10
+        # Produce 1000 graphs in which the majority of nodes are very likely assigned to
+        # community 1, thus approximate
+        # 1000 ER graphs with probability almost equal to 1.
+        graphs = []
+
+        for i in range(300):
+            graphs.append(mmsbm(n, p, alpha, directed=True, loops=True, rng=rng))
+        graphs = np.stack(graphs)
+
+        # check that probability of these graphs is actually almost equal to that of the
+        # first block which is 1.
+        self.assertAlmostEqual(np.mean(graphs), p[0][0], delta=0.01)
+
+        alpha = [0.05, 1000]
+        # Produce 1000 graphs in which the majority of nodes are very likely assigned to
+        # community 2, thus approximate
+        # 1000 ER graphs with probability almost equal to 0.7.
+        graphs = []
+
+        for i in range(300):
+            graphs.append(mmsbm(n, p, alpha, directed=True, loops=True, rng=rng))
+        graphs = np.stack(graphs)
+
+        # check that probability of these graphs is actually almost equal to that of the
+        # second block which is 0.7.
+        self.assertAlmostEqual(np.mean(graphs), p[1][1], delta=0.01)
+        pass
+
+    def test_labels(self):
+        rng = np.random.default_rng(self.seed)
+        np.random.seed(self.seed)
+        alpha = np.array([0.0001, 100])  # assign all nodes to second group(index 1)
+        A = mmsbm(
+            self.n,
+            self.p,
+            alpha,
+            rng=rng,
+            directed=False,
+            loops=False,
+            return_labels=True,
+        )
+        labels = A[1]
+        expected_labels = np.ones((self.n, self.n))
+        np.fill_diagonal(expected_labels, -1)
+        # since we expect all nodes on the off diagonal to pertain to community 2
+        # we expect all values on the off diagonal to be 1
+        # We expect nan on the diagonal as this graph has no loops
+
+        # check that expected labels is the same as the labels output by function
+        self.assertTrue(np.allclose(labels, expected_labels))
+        pass
+
+    def test_loop_directed(self):
+        rng = np.random.default_rng(self.seed)
+        A = mmsbm(self.n, self.p, self.alpha, rng=rng, directed=True, loops=True)
+
+        # check loops and directed
+        self.assertFalse(is_symmetric(A))
+        self.assertFalse(is_loopless(A))
+
+        # check dimensions
+        self.assertTrue(A.shape == (self.n, self.n))
+        pass
+
+    def test_noloop_undirected(self):
+        rng = np.random.default_rng(self.seed)
+        # Test that when loops = False and directed = False the output is undirected
+        # and with no loops
+        A = mmsbm(self.n, self.p, self.alpha, rng=rng, directed=False, loops=False)
+
+        # check loopless and undirected
+        self.assertTrue(is_symmetric(A))
+        self.assertTrue(is_loopless(A))
+
+        # check dimensions
+        self.assertTrue(A.shape == (self.n, self.n))
+        pass
+
+    def test_bad_inputs(self):
+        with self.assertRaises(ValueError):
+            n = -1  # n must be greater than 0
+            mmsbm(n, self.p, self.alpha, rng=self.rng)
+
+        with self.assertRaises(TypeError):
+            n = "1"  # n must be an integer
+            mmsbm(n, self.p, self.alpha, rng=self.rng)
+
+        with self.assertRaises(TypeError):
+            n = 2.5  # n must be an integer
+            mmsbm(n, self.p, self.alpha, rng=self.rng)
+
+        with self.assertRaises(TypeError):
+            n = ["1", 10]  # n must be an integer
+            mmsbm(n, self.p, self.alpha, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            p = 0.5  # p must be a 2d array, not scalar
+            mmsbm(self.n, p, self.alpha, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            p = [[0.2, 0.1], ["str", 0.6]]  # p must only contain mumeric elements
+            mmsbm(self.n, p, self.alpha, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            p = [[5, 5], [4, 4]]  # p must be between 0 and 1
+            mmsbm(self.n, p, self.alpha, rng=self.rng)
+
+        with self.assertRaises(TypeError):
+            alpha = 1  # alpha must be an array
+            mmsbm(self.n, self.p, alpha, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            # test deafult alpha = None, should output ValueError
+            mmsbm(self.n, self.p, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            alpha = ["str", 0.3]  # alpha must only contain numeric elements
+            mmsbm(self.n, self.p, alpha, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            alpha = [0.1, -2]  # alpha entries must be > 0
+            mmsbm(self.n, self.p, alpha, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            alpha = [
+                0.1,
+                0.5,
+                0.7,
+            ]  # dimension of p is 2x2 so alpha should be of length 2.
+            mmsbm(self.n, self.p, alpha, rng=self.rng)
+
+        with self.assertRaises(TypeError):
+            rng = 2  # must be generator
+            mmsbm(self.n, self.p, self.alpha, rng=rng)
+
+        with self.assertRaises(TypeError):
+            loops = 2  # loops must be a bool
+            mmsbm(self.n, self.p, self.alpha, loops=loops, rng=self.rng)
+
+        with self.assertRaises(TypeError):
+            directed = 2  # directed must be a bool
+            mmsbm(self.n, self.p, self.alpha, directed=directed, rng=self.rng)
+
+        with self.assertRaises(ValueError):
+            # specified undirected but provided directed B matrix
+            p = np.vstack(([0.6, 0.2], [0.3, 0.4]))
+            mmsbm(self.n, p, self.alpha, directed=False, rng=self.rng)
+
+        with self.assertRaises(TypeError):
+            return_labels = 2  # return labels must be a bool
+            mmsbm(self.n, self.p, self.alpha, return_labels=return_labels, rng=self.rng)