Active Learning of Convex Halfspaces on Graphs

1 Introduction . We systematically characterise active learning of geodesically convex halfspaces on graphs . While unlabelled graphs such as social networks are readily accessible , obtaining many labels is tedious and labour intensive . Active learning reduces the labelling effort by iteratively selecting informative data points to be labelled . Graph-based active learning , in particular , has been successfully applied on communication [ Regol et al. , 2020 ] and protein-protein-interaction networks [ Vazquez et al. , 2003 , Xiong et al. , 2014 ] . While most previous approaches assume that the number of edges with differently labelled vertices , the cut size , is small [ Afshani et al. , 2007 , Guillory and Bilmes , 2009 , Dasarathy et al. , 2015 ] , we take a different approach and instead assume geodesic convexity : For every pair of vertices with the same label it holds that every vertex on every shortest path between them also has that label . This assumption is used , for example , by biologists on gene similarity networks [ Zhou et al. , 2002 ] and cancer-related protein-protein-interaction networks [ Li et al. , 2012 , 2013 ] . It also typically holds for connected subgraphs of collaboration networks [ Marc and Šubelj , 2018 , Šubelj et al. , 2019 ] and our preliminary experiments confirmed that many communities in community detection datasets are convex . We derive bounds on the number of queries required to determine all labels , that is , the query complexity , using concepts from convexity theory and geodesic convexity spaces , which naturally generalise the regular Euclidean convexity . While learning convex sets and halfspaces in Euclidean spaces has been intensively investigated in the machine learning community for over half a century , geodesic convexity on graphs is understudied . For a concise presentation of our theoretical results in this paper , we concentrate on active learning of halfspaces on graphs , that is , the vertex set is partitioned into two classes which are both geodesically convex . After introducing the concepts of graph and abstract convexity theory in Section 2 , we derive general label-independent upper and lower bounds on the query complexity . We discuss that such general bounds can be loose on specific graphs and derive tighter lower bounds along the separation axioms of abstract convexity theory in Section 3 . This allows us to substantially reduce the gap between our upper and lower bounds . Note that Euclidean spaces can not be structured in a similar way : while the separation axioms partition the space of all graphs into non-empty sets , all separation axioms 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . hold in all real vector spaces including the Euclidean spaces . We discuss related topics like previous active learning bounds that depend on the cut size between the classes , active learning of halfspaces in Euclidean spaces , non-active learning of geodesically convex halfspaces , and other related learning theoretic concepts in Section 4 . Notably , we provide a bound on the VC dimension of geodesically convex halfspaces given by the Radon number . After presenting an empirical validation of our assumptions and approach in Section 5 , we conclude with a discussion of future work . 2 Convexity spaces . In this section , we introduce the required concepts of convexity theory . For a more thorough introduction on convexity theory we refer the reader to van de Vel [ 1993 ] and Pelayo [ 2013 ] . For a set X and a family C ⊆ 2X of subsets , the pair ( X , C ) is a convexity space if ( i ) ∅ , X ∈ C , ( ii ) C is closed under intersection , and ( iii ) C is closed under unions of sets totally ordered by inclusion . For finite set systems , property ( iii ) always holds . Any set in C is called convex . If a set C and its complement X \ C are convex , both are called halfspaces . Two disjoint sets A , B are halfspace separable if there exists a halfspace C such that A ⊆ C and B ⊆ X \C . A mapping σ : 2X → 2X is a convex hull ( or closure ) operator if for all A , B ⊆ X with A ⊆ B ( i ) σ ( ∅ ) = ∅ , ( ii ) σ ( A ) ⊆ σ ( B ) , ( iii ) A ⊆ σ ( A ) , and ( iv ) σ ( σ ( A ) ) = σ ( A ) . Any convexity space ( X , C ) induces a convex hull operator by σ ( A ) = ⋂ { C | A ⊆ C ∈ C } . A set A ⊆ X is convex , that is A ∈ C , if and only if is equal to its convex hull , A = σ ( A ) . A set H ⊆ X is a hull set if its convex hull is the whole space , σ ( H ) = X . For A , B ⊆ X , the set A/B = { x ∈ X | A ∩ σ ( B ∪ { x } ) 6= ∅ } is the extension of A away from B . For a , b ∈ X , the extension { a } / { b } is also called a ray a/b . Two disjoint sets A1 , A2 form a partition of A ⊆ X if A1 ∪ A2 = A . The partition A1 , A2 of A is a Radon partition if σ ( A1 ) ∩ σ ( A2 ) 6= ∅ . The Radon number is the minimum number r such that any subset of X of size r has a Radon partition . A particular type of convexity is an interval convexity . Apart from the convex hull σ ( · ) it has an interval mapping I : X×X → 2X such that for all x , y ∈ X , ( i ) x , y ∈ I ( x , y ) and ( ii ) I ( x , y ) = I ( y , x ) . I ( x , y ) is the interval between x and y . We denote I ( A ) = ⋃ a , b∈A I ( a , b ) . A set C in an interval convexity space is convex if and only if C = I ( C ) . The convex hull is given by σ ( A ) = ⋃∞ k=1 I k ( A ) , where I1 ( · ) = I ( · ) and Ik+1 ( · ) = I ( Ik ( · ) ) , A well-known instance of interval convexity spaces are metric spaces ( X , d ) . There , the interval contains all the points for which the triangle inequality holds with equality : Id ( x , y ) = { z ∈ X | d ( x , y ) = d ( x , z ) + d ( z , y ) } . In Euclidean space this corresponds to all points on a line segment and leads to the classical notion of convex sets . We study metric spaces induced by graphs . The geodesic convexity ( or shortest path convexity ) of a connected graph G = ( V , E ) is given by the interval mapping Id , where d : V 2 → R is the shortest path distance in G. Let x , y ∈ V . For unweighted graphs d ( x , y ) is the minimum number of edges on any x-y-path and for graphs with edge weights , w : E → R > 0 , it is the minimum total edge weight of any x-y-path . A set of vertices C ⊆ V is , thus , ( geodesically ) convex if and only if C contains every vertex on every shortest path joining vertices in C , corresponding again to the Euclidean case . We denote by V ( G ) the vertex set of a graph G , the Radon number of the induced geodesic convexity space of a graph as r ( G ) , and the size of the minimum hull set as h ( G ) . Rays a/b in graphs can be defined using the shortest path distance d : In unweighted graphs a/b = { v ∈ V | d ( a , v ) < d ( b , v ) } and in weighted graphs a/b = { v ∈ V | d ( b , v ) = d ( b , a ) + d ( a , v ) } . A vertex v is extreme if { v } forms a halfspace , that is , V \ { v } is convex . We denote the set of extreme vertices of G as Ext ( G ) . In unweighted graphs , v is extreme if and only if its neighbours form a clique . The diameter d ( G ) of a weighted or unweighted graph G is the maximum number of edges in any shortest path in G . 3 Active learning halfspaces on graphs . Having introduced the necessary concepts from convexity theory , we now present the main theoretical results of the paper . In active node classification , a graph G = ( V , E ) with unknown vertex labels λ : V → { 0 , 1 } is given and the goal is to accurately predict all labels using as few as possible vertex queries . Edges can be weighted or unweighted . The learner queries the vertices one by one : it iteratively selects vertices v ∈ V and receives their label λ ( v ) . In this paper , we consider active learning on an undirected connected graph where the vertices with same label form geodesically convex sets . The classes are , thus , halfspaces and we call the labelling halfspace separable . Our results in this section are upper and lower bounds on the number of queries required to deduce all labels λ of the graph . We denote the number of queries required to identify any halfspace separable labelling in the worst-case , the query complexity , by qc ( G ) . We derive two general upper bounds ( Proposition 1 , Theorem 5 ) and one lower bound ( Theorem 7 ) on qc ( G ) that hold for any halfspace separable labelling . To reduce the gap between the upper and lower bounds ( Proposition 8 ) , we derive increasingly tighter lower bounds along separation axioms ( Theorem 10 ) . In this section , we consider only undirected and connected graphs . Full proofs and a discussion of the multiclass , directed , and disconnected setting are in the supplementary material . 3.1 Upper bounds on the query complexity . To derive a simple upper bound , we note that one immediate consequence of the halfspace assumption is that any shortest path P can have at most one cut edge , that is , an edge with differently labelled endpoints . To deduce all labels of P , we first query its endpoints . If they have the same label , we know that all vertices on P have this label . Otherwise , we find the cut edge by binary search with at most dlog d ( G ) e queries , as the length of P is at most the diameter |V ( P ) | − 1 ≤ d ( G ) . Here , log is the base 2 logarithm . We can generalise this approach to the whole graph using shortest path covers [ Thiessen and Gärtner , 2020 ] , which is a set S of shortest paths whose vertices cover the graph : ⋃ P∈S V ( P ) = V ( G ) . Performing binary search on each path in S gives our first upper bound . Proposition 1 . For any weighted graph G with minimum shortest path cover S∗ the query complexity can be bounded as qc ( G ) ≤ |S∗| ( 2 + dlog d ( G ) e ) . By considering a complete graph with edge weights such that the position of the cut edges on all shortest path in the minimum shortest path cover can be chosen independently of each other , we can show that this bound is tight : Proposition 2 . For any ` , s ∈ N , there exists a weighted graph G with diameter d ( G ) = ` and minimum shortest path cover S∗ of size s such that qc ( G ) ≥ |S∗| log d ( G ) . However , the graph in Proposition 2 is an artificial worst-case example . For most graphs , we can do much better as the labels of vertices on different paths in the cover can typically not be chosen independently of each other . Consider a cubic hypergrid graph G , that is , the Cartesian product of k paths with ` vertices each . More than |V ( G ) |/d ( G ) + 1 ≥ ` k/k ` shortest paths are needed to cover G but 2 + dlog ( k ` ) e queries suffice to identify all labels , as fixing one cut edge determines all others . Our main idea to improve the bound and derive a better algorithm for active learning of geodesically convex halfspaces is to deduce additional labels using convex hulls and extensions after each query . Algorithm 1 realises this idea . It first queries the vertices of a hull set ( line 1 ) . If all labels in the hull set are the same , we conclude that all vertices in the graph have the same label ( line 2-3 ) . Otherwise , it performs binary search on a shortest path between two vertices with different labels to identify a cut edge { a , b } ( line 4-5 ) . Finally , Algorithm 1 queries the remaining vertices . It initialises the sets A and B with the convex hull of the rays a/b and b/a , respectively , corresponding to vertices with already known labels ( line 6 ) . In each iteration of the main loop , the algorithm queries a vertex v in the set Ŵ=ab = V ( G ) \ ( σ ( a/b ) ∪ σ ( b/a ) ) , consisting of all vertices whose labels we can not directly deduce through convex hulls and extensions , and updates the sets A and B using the new vertex ( line 7-12 ) . The number of queries spend in the first five lines are summarised in Lemma 3 . Lemma 3 . Let G be a weighted graph with halfspace separable labels . Using h ( G ) + dlog d ( G ) e queries , we can either find a cut edge or determine that all vertices of the graph have the same label . Proof . The convex hull of any minimum hull set is the whole graph . If all vertices in a hull set have the same label , then all vertices in the graph have this label . If not , we can take two vertices with different labels and find a cut edge on a shortest path between them with dlog d ( G ) e queries . Lemma 4 shows that the initialisation and updates of A and B in Algorithm 1 are indeed valid . Lemma 4 . Let G be a weighted graph with halfspace separable labels given by a halfspace C and let A ⊆ C and B ⊆ V ( G ) \ C. It holds that σ ( A/B ) ⊆ C. Proof sketch . A/B ⊆ C as A ∩ σ ( B ∪ { x } ) ⊆ C ∩ ( V ( G ) \ C ) = ∅ for all x ∈ V ( G ) \ C. This implies σ ( A/B ) = ⋂ { C ′ | A/B ⊆ C ′ , C ′ convex } ⊆ C by definition . Algorithm 1 : Ray-based Active Halfspace Learning on Graphs Input : graph G , oracle access to labels λ Output : halfspaces corresponding to both classes 1 Compute a hull set H and query its vertices 2 if ∀h , h′ ∈ H : λ ( h ) = λ ( h′ ) then 3 return ( ∅ , V ( G ) ) 4 Choose h , h′ ∈ H such that λ ( h ) 6= λ ( h′ ) 5 Perform binary search queries on any shortest h-h′-path to find a cut edge { a , b } 6 A : = σ ( a/b ) , B : = σ ( b/a ) 7 while A ∪B 6= V ( G ) do 8 query any vertex v ∈ V ( G ) \ ( A ∪B ) 9 if λ ( v ) = λ ( a ) then 10 A : = σ ( ( A ∪ { v } ) /B ) , B : = σ ( B/ ( A ∪ { v } ) ) 11 else 12 A : = σ ( A/ ( B ∪ { v } ) ) , B : = σ ( ( B ∪ { v } ) /A ) 13 return ( A , B ) We can bound the remaining number of vertices queried by Algorithm 1 using the following set W ∗=ab = { v ∈ Ŵ=ab | @ w ∈ Ŵ=ab \ { v } such that v ∈ ( w/a ) ∩ ( w/b ) } as it is enough to know the labels of the vertices in W ∗=ab to deduce all labels of Ŵ=ab . Theorem 5 shows that Algorithm 1 is correct and summarises our main upper bound . Theorem 5 . Let G be a weighted graph . For the query complexity it holds that : qc ( G ) ≤ h ( G ) + dlog d ( G ) e+ max { a , b } ∈E ( G ) |W ∗=ab| . Proof sketch . By Lemma 3 , we can use h ( G ) + dlog d ( G ) e queries to find the first cut edge { a , b } . Every time Algorithm 1 queries a vertex v ∈ Ŵ=ab \W ∗=ab there is a w ∈W ∗=ab by definition such that v ∈ ( w/a ) ∩ ( w/b ) . It holds that w /∈ A∪B , as otherwise v ∈ ( w/a ) ∩ ( w/b ) ⊆ A∪B , which contradicts line 8 of Algorithm 1 , v ∈ V ( G ) \ ( A ∪ B ) . Thus , in each iteration at least one new vertex from W ∗=ab will be added to A or B . This bounds the number of iterations by |W ∗=ab| . Comparing our two upper bounds , we see that Theorem 5 is preferable to Proposition 1 , as long as |W ∗=ab| is small . To see this , note that the endpoints of paths in a minimum shortest path cover S∗ form a hull set , implying h ( G ) ≤ 2|S∗| . If |W ∗=ab| is small , the bound of Theorem 5 is additive in the dominating terms h ( G ) + dlog d ( G ) e ≤ 2|S∗|+ dlog d ( G ) e , not multiplicative as in Proposition 1 . To state the bound for the unweighted case in more common parameters , we relate |W ∗=ab| to the well-studied treewidth tw ( G ) [ Bodlaender , 1996 ] , which measures the ‘ tree-likeness ’ of a graph . It is known , that the largest k ∈ N , such that the complete bipartite graph K2 , k is a minor of a graph G , is at most twice the treewidth [ Bodlaender et al. , 1997 ] : k ≤ 2 tw ( G ) . This bounds the size |W ∗=ab| ≤ 2 tw ( G ) for any edge { a , b } , as the vertices { a , b } ∪W ∗=ab form a minor K2 , |W∗=ab| . Consequently , graphs with small treewidth like molecules [ Horváth and Ramon , 2010 ] and infrastructure-based networks [ Maniu et al. , 2019 ] have small W ∗=ab . Corollary 6 summarises this result . Corollary 6 . Let G be an unweighted graph . For the query complexity it holds that qc ( G ) ≤ h ( G ) + dlog d ( G ) e+ 2 tw ( G ) .

Summary: The authors study the problem of  learning geodesically convex halfspaces on graphs within an active learning setting. They propose a new algorithm and rigorously analyzed it by showing lower and upper bounds of the query complexity. They also carry out an experimental evaluation to validate their theoretical results.