Package: connectivity-modifier
cmConnectivity Modifier (CM) is a generic meta-method for community detection while ensuring a certain connectivity
(minimum number of edges to remove to disconnect a community) on the output communities (clusters). To be more precise, suppose that you want to ensure that Leiden clusters must not be "easily-cut". For example, ensuring that none of the output clusters have connectivity below
$\log_{10}(n)$, $n$ the size of any cluster, you can run CM paired with Leiden, and it will ensure that all output clusters have that minimum size cut. CM supports customizable requirements on the connectivity of the clusters. CM currently supports Leiden (CPM optimization), IKC, and Leiden (modularity optimization) out-of-the-box. After installing the necessary dependencies, users can simply run CM to obtain Leiden clusters with strong guarantees on connectivity.InstallationOur software is provided via PyPI and supports at least Python 3.9.pip3 install --pre connectivity-modifier # install prerelease versionNote that you must installVieCutas a dependency explicitly. More specifically,mincutthe binary must be compiled and the path tomincutmust be specified in the config file (explained below).Say that you installedmincutas/foo/bar/mincut; then you want to create a config file in~/.config/cm/config.tomland have something like this:[tools]ikc_path="{project_root}/third_party/ikc.py"# {project_root} is a specific path resolving to the source code rootleiden_path=""# this config option is currently unusedviecut_path="/foo/bar/mincut"# viecut's pathAlso feel free to use this alternativeCMakeLists.txtfile if having issues with compiling VieCut using their built-in CMakeLists (e.g. having issues with MPI, etc.).If the main executable detects thatcm.tomlis in the current working directory, thecm.tomlconfig file will have the highest priority instead.After all this, trycm --help, and you should see something like this:Usage:cm[OPTIONS]Connectivity-Modifier(CM).Takeanetworkandclusteritensuringcutvalidity

Options:-i,--inputTEXT[required][...]UsageOur main executable is provided ascm, and we list the options below:-i, --input GRAPH_TSVThe input graph to be clustered, wheregraph.tsvis a tab-delimited edgelist, only including integer edge ids. Note that we follow theigraphconvention, where we assume that the input node ids are continuous, and if not, dummy nodes are added.-c, --clusterer [leiden|ikc|leiden_mod]The clusterer to be paired with. If using with an existing clustering (-e), then the same clusterer must be used (see below). Otherwise, one must decide which clusterer should be used. The clusterers are:leiden: Leiden (leidenalg) with CPM optimization, must specify-g, --resolutionlaterikc: Iterative k-core, must specify-klaterleiden_mod: Leiden with modularity optimization, no other parameters allowed to be specified-e, --existing-clustering CLUSTERING_FILESpecifies the starting clustering (in effect saving time forcmto reproduce the initial clustering) to bemodifiedto have sufficient connectivity thresholds (c.f.-t). The file format is "native" to the clustering method. For example, for IKC, it is the default IKC csv output format. For Leiden, it is the Leiden output format (i.e., tab-delimited node_id cluster_id file).-g, --resolution FLOAT,-k, --k INTEGERThe respective parameters for either Leiden(CPM) (-c leiden) or IKC (-c ikc). Only at most one should be specified, and for modularity optimization neither should be specified.-o, --output OUTPUT_PREFIXThe output prefix. Two files will be produced, first theOUTPUT_PREFIXwill have a file denoting the last cluster a node has been in, and{OUTPUT_PREFIX}.tree.jsonis a serialized tree denoting the history of the execution of the algorithm. See alsoconverting the output to more parsable formats.-t, --threshold TEXTThreshold expression.cmguarantees that the output clustering all have clusters that are above a specific threshold. We list some examples for-tbelow:# each line denotes a valid example for "-t"2# connectivity must > 20.1mcd# connectivity must > 0.1 MCD, MCD the minimum intra-cluster degree0.1mcd+42# linear combinations are allowed to some extent1log10# > log10 of size of cluster99log10+0.0002mcd+1# combinations like this are allowed-d, --working-dir TEXTOptional for throw-away runs; specifies wherecmshould store its temporary files. Prudently one
should always specify different working directories for different runs under the same directory.--ignore-trees&--ignore-smaller-than INTEGERAllows CM to skip processing trees (those clusters with nodes equals number of edges plus one) and
those clusters that are smaller than the specified number. These clusters will be skipped for processing
but included in the output.Example commands# Leiden, CPM optimization (resolution = 0.1)# BUT, the output clusters must satisfy global connectivity > 1 * log10(n), n the size of clustercm-igraph.tsv-cleiden-g0.1-t1log10-oleiden_clus.txt
cm2universal-ggraph.tsv-ileiden_clus.txt-oleiden_clus.txt# generates two files: leiden_clus.txt.before.json and leiden_clus.txt.after.json# See also "Format Conversion"# IKC, k = 10# BUT, the output clusters must satisfy global connectivity > 0.1 * mcd, MCD the minimum intra-cluster degree among all nodes# we additionally use an existing IKC clustering (ikc_output.csv) as the starting point to be modifiedcm-igraph.tsv-cikc-k10-t0.1mcd-eikc_output.csv-oikc_clus.txtNotethat for serious runs,-dshould also be specified to avoid overwriting temporary files.Format ConversionThe default output ofcmcontains the entire history of the execution of the algorithm, i.e., not the true usual clustering format. This format allows preservation of much information, but often times for data analysis, only knowing the clusteringbeforemodifying the connectivity (i.e., as if just running the base method) andaftermodifying the connectivity is enough. These two sets of clusters can be obtained fromcmusing the specialized toolcm2universal(which comes included with the project):# INPUT_GRAPH is the same INPUT_GRAPH# CM_OUTPUT_PREFIX is the same output prefix of `cm`, i.e., `{CM_OUTPUT_PREFIX}.tree.json` and `CM_OUTPUT_PREFIX` are existing files# CLUSTERS_OUTPUT_PREFIX is where you want the converted clusterscm2universal-gINPUT_GRAPH-iCM_OUTPUT_PREFIX-oCLUSTERS_OUTPUT_PREFIXTwo files will be generated:{CLUSTERS_OUTPUT_PREFIX}.before.jsonand{CLUSTERS_OUTPUT_PREFIX}.after.json, containing the original and after clusters respectively. Thejsonfiles use the so-called "universal" new-line delimited JSON format, looking like this:{"label":"0","nodes":[0,3,7,9],"connectivity":1}{"label":"46","nodes":[5765736,4717164,14154348,3144303,6290035,3668596,1571445,2620022,4717176],"connectivity":2}These files can be directly parsed (each line is a cluster,labelthe cluster name,nodesthe node ids of that cluster,connectivitythe edge connectivity) or can be paired with the data science toolBelinda.DevelopmentWe usePoetryto manage our progress and follow the Poetry conventions. See below for some example commands:poetryinstall# install networkit and copoetryrunpytest# run testsPseudocodeAssuming that we have an existing clustering method $f$ that takes in a graph and outputs a list of clusters, our algorithm roughly does the following:Obtain clusters ${S_i}$ from $f$ if not provided in the input.Initiate astack$Q$ containing all above clusters (the stack meant that everything is done in a DFS fashion)When $Q$ non-empty, take $S_i$ from $Q$. Do the followingCalculate connectivity $\lambda$ for $S_i$Done viamincut -b -s -o [mincut_output_path] [input_graph_path] cactus(wheremincutis provided by Viecut)If $\lambda > t$ for $S_i$, where $t$ is as specified by the-tparameter, then $S_i$ is marked for output, and we never touch $S_i$ againOtherwise. $S_i$ is split into $S_i^a$ and $S_i^b$ by the minimum cut, and $f(S_i^a)$ and $f(S_i^b)$ are added to $Q$.