Title: A  Single-Swap Local Search Algorithm for k-Means of Lines
Venue: NeurIPS 2025
Accepted: yes
Keywords: local search，line clustering，k-means of lines
Abstract: Clustering is a fundamental problem that has been extensively studied over past few decades, with most research focusing on point-based clustering such as $k$-means, $k$-median, and $k$-center.
However, numerous real-world applications, such as motion analysis, computer vision, and missing data analysis, require clustering over structured data, including lines, time series and affine subspaces (flats), where traditional point-based clustering algorithms often fall short.
In this paper, we study the $k$-means of lines problem, where the input is a set $L$ of lines in $\mathbb{R}^d$, and the goal is to find $k$ centers $C$ in $\mathbb{R}^d$ such that the sum of squared distances from each line in $L$ to its nearest center in $C$ is minimized. The local search algorithm is a well-established strategy for point-based $k$-means clustering, known for its efficiency and provable approximation guarantees. However, extending local search algorithm  to the $k$-means of lines problem is nontrivial, as the capture relation used in point-based clustering does not generalize to the line setting. This is because that the point-to-line distance function lack the triangle inequality property that supports geometric analysis in point-based clustering. Moreover, since lines extend infinitely in space, it is  difficult to identify effective swap points that can significantly reduce the clustering cost. To overcome above obstacles, we introduce a *proportional capture relation* that links optimal and current centers based the assignment proportions of lines, enabling a refined analysis that bypasses the triangle inequality barrier. We also introduce a *CrossLine* structure, which provides a principled discretization of the geometric space around line pairs, and ensures coverage of high-quality swap points essential for local search, thereby enabling effective execution of the local search process. Consequently, based on the proposed components, we develop the first single-swap local search algorithm for the $k$-means of lines problem, achieving a $(500+\varepsilon)$-approximation in polynomial time for low-dimensional Euclidean space.


Peer reviews:
comments: SUMMARY:
The paper introduces an algorithm for clustering lines into k sets. The method is reminiscent of K-means++, but instead of sampling centers from a distribution, the authors discretize the continuous space using a grid and search over a finite set of candidate points. The theoretical analysis is solid and provides meaningful guarantees. I didn’t go through every proof in detail, but the parts I did check seemed correct, and the overall reasoning appears sound.

One limitation is the absence of numerical experiments—not even simple ones on synthetic data. While the authors mention that the method isn’t expected to be robust to noise, it would still be helpful to see the algorithm in action to better understand its behavior. The paper is clearly written, and although it’s more theoretical than most NeurIPS submissions, I think it’s a worthwhile contribution. That said, theory papers can be difficult to fully evaluate under conference reviewing timelines, and small technical issues might go unnoticed.

STRENGTHS AND WEAKNESSES:
Strengths:
>Clear and well-organized theoretical analysis.
>Tackles a relatively unexplored problem.
>The algorithm design is interesting and non-trivial.

Weaknesses:
>No experimental validation.
>No discussion of robustness or practical performance.

QUESTIONS:
Could you explain why you chose not to include any numerical experiments?
Even basic examples—successful or not—would help make the paper more accessible to a broader audience.

LIMITATIONS:
Lack of empirical evaluation.
The analysis is based on idealized data, with no consideration for noise, although this is addressed in the discussion.
Potential sensitivity to initialization or local optima is not addressed.
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-06-28

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the $k$-means clustering problem, where the input objects are lines in Euclidean space rather than points. The authors propose a 500-approximation algorithm using a local search framework, which is adapted in a non-trivial way to handle the geometric and analytical challenges posed by clustering lines.\n\nThe algorithm proceeds in two main steps:\n1. Initialization via coreset: It begins with a coreset-based algorithm from [Marom and Feldman (2019)], originally developed for the $k$-median of lines. Although this coreset was designed for a different objective, the authors show that it yields a constant-factor approximation for the $k$-means of lines problem—though the precise approximation ratio is not specified.\n2. Local search: The solution is then improved using a standard local search approach—specifically, center swapping—adapted to the line setting. The candidate centers are constructed by sampling two lines with probability proportional to their current cost and creating a fine grid near their intersection point. This process is repeated a polynomial number of times.\n\nA major analytical challenge is that a single line can be simultaneously close to multiple centers, even if those centers are far apart from each other, making it difficult to directly apply the triangle inequality. Furthermore, since centers can be located anywhere in continuous space, the analysis requires careful discretization.\n\nThe high-level argument behind the approximation guarantee is that if the current solution is significantly worse than the optimum (e.g., more than 500× the optimal cost), then there’s a good chance that the weighted-sampled lines will be close to optimal centers. If these lines are near the same optimal center, then their intersection is a promising candidate for placing a new center. This allows the algorithm to iteratively improve the solution by exploring neighborhoods around such intersections.\n\nSTRENGTHS AND WEAKNESSES:\n### Strength\n- This is the first constant-factor approximation algorithm for the $k$-means of lines problem with an explicit approximation ratio.\n- It successfully adapts the local search technique to this geometric setting, which involves non-trivial modifications and theoretical insights.\n\n### Weakness\n- The analysis, especially the main proof, is difficult to follow. Breaking it down into smaller, modular lemmas and including more illustrative diagrams would significantly improve readability.\n- Due to the complexity and density of the arguments, I was not able to fully verify all parts of the analysis. While the general structure seems plausible, a more accessible presentation would increase confidence in the result.\n\nQUESTIONS:\n### Major comments\n- Assumption on non-parallel lines: The paper assumes all input lines are pairwise non-parallel. In practical settings (e.g., road networks, trajectories), parallel lines are common. Is there a simple extension of your algorithm to handle such inputs?\n- Radius parameter in CrossLine structure: The definition relies on a radius $r$ tied to the optimal cost, which is unknown in practice. How is $r$ chosen algorithmically? Can it be estimated in a data-dependent or iterative way?\n- Candidate center construction: Lemma 8 suggests that optimal centers are near intersections of two nearby lines. Since there are only $O(n^2)$ such intersections, could you simply use all of them as candidate centers for classical point-based local search? Would this simplify the algorithm or analysis?\n\n### Minor comments\n- Line 67–68: The description appears to misrepresent paper [9].\n- Line 70–72: The summary of paper [10] does not seem accurate.\n- Line 200–202: Citation [22] does not pertain to local search.\n- Line 211: “near to $c^*$” → “near $c^*$”\n- Line 210–213: The claim about the failure of capture relations in the line setting is not entirely clear. A visual counterexample would help clarify this point.\n- Line 231–237: Regarding type-2 matched swap pairs: if a center $c$ captures no optimal centers, you pair it with an optimal center “borrowed” from a richer center $c’$. What becomes of $c’$—is it then a type-1 or type-2 pair? This edge case deserves clarification.\n- Line 250 (Definition 2): Please clarify the notion of “reassignment cost”—is this just an upper bound on the cost increase when removing a center?\n- Lines 110, 270, 274, 292: Typo—“closed to” → “close to”\n- Line 292: The inequality $\\Delta(L(\\psi_S(c)),C) \\geq 9\\cdot \\Delta (L(\\psi_S(c)),\\{\\psi_S(c)\\})$ based on Definition 3 is not immediately obvious. Please expand on how Definition 3 implies this bound.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper studies the $k$-means of lines problem. Compared to the original $k$-means problem, the dataset $P$ consists of $n$ lines rather than points. The algorithm needs to find $k$ centers $C \subset \mathbb{R}^d$ to minimize $\sum_{p\in P}\min_{c\in C}d(p,c)^2$, where $d(p,c)$ denotes the distance between line $p$ and point $c$. 

This paper proposes the first local search algorithm (with a theoretical guarantee) for the $k$-means problem. Local search is widely used for designing efficient clustering algorithms. However, it is hard to generalize the local search algorithms for the original $k$-means problem to the line setting, since the geometric properties of lines are very different from points. 

In order to overcome the challenges, the authors introduce a new mapping between the current solution and the optimal solution. 
They also designed a new sampling procedure which discretizes the solution space into a finite set of representative points so that they can sample candidate centers efficiently.

STRENGTHS AND WEAKNESSES:
Strength:

1. This paper proposes the first constant-approximate algorithm for the $k$-means problem in polynomial time. In order to apply local search to the $k$-means of lines problem, the authors develop several new techniques to overcome the complicated geometric properties of lines. These techniques may also be used in the following studies of $k$-means of lines or even other geometric problems. 

2. Their algorithm is quite simple and efficient in practice (in low-dimensional cases). 

Weakness:

The running time of the algorithm has an exponential dependency on $d$, which means the algorithm might not be polynomial time in high-dimensional cases.

QUESTIONS:
See weakness.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 2
ORIGINALITY: 4
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of clustering a set of lines using points as cluster centers and the k-means objective. The idea is to develop an approximation algorithm in the framework of local swaps. The main difficulty is that the point-to-line distance, used in the paper, does not satisfy the triangle inequality, and thus standard local-swap arguments are not amenable to this setting. Another question to address for the method to be operational is the initialization step, for initializing cluster centers, before the local-swap phase. The paper proposes a new algorithm that addresses those questions and demonstrates that the proposed algorithm achieves a constant factor approximation in this local-swap setting. The paper is of mostly theoretical nature and there is no experimental evaluation of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nThe main strengths of the paper are the clarity in the problem definition, the rigorous analysis of the proposed method, and the achievement of obtaining a constant-factor approximation for the problem of clustering lines. \nOn the other hand, I found a few important weaknesses. \nW1. I found the motivation of the problem very weak. The paper states as motivation applications in traffic and motion data analysis. However, such real-world data are never straight (infinite lines). Another element missing motivation is the choice of using points as cluster centers. Why not using lines as cluster centers? This looks a more straightforward generalization of the k-means setting for points, where both the data and the cluster centers are points. Overall, I am totally on board that the problem is intellectually interesting and a meaningful generalization of the k-means problem for points, however, I think that for a AI/ML conference like NeurIPS a stronger motivation is needed. Perhaps the paper is a better fit for a venue in theoretical computer science, or computational geometry.\nW2. Earlier work provides a PTAS for the same problem, while this paper provides a constant-factor approximation, with a very large constant. Thus, this is not the theoretically best result available. Indeed, the claim of the paper is that this is the first constant-factor approximation for the local-swaps setting, not for the problem itself. It is also true that the PTAS algorithm has high complexity. However, to strengthen the value of the paper as a practical alternative with manageable complexity I would expect to see some experimental evaluation of the proposed method. Additionally, a convincing experimental evaluation would also address the issue of motivation in real-world applications, by forcing the authors to identify meaningful datasets and applications to use the lines-clustering problem and methods. Overall, I appreciate the contribution of \"first constant-factor approximation within a specific solution framework\" and I value the technical depth and rigorous analysis of the paper, however, in addition to the weak motivation, and lack of experimental evaluation, I think that the overall contribution is not sufficient for NeurIPS.\n\nQUESTIONS:\nI welcome answers to weak points W1 and W2 mentioned above.\n\nLIMITATIONS:\nThe authors make an honest attempt to list the limitations of their problem setting. In addition to the ones listed by the authors, I would also add the weak points mentioned above.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a local search (LS) algorithm for k-means clustering of lines, where the input are lines in R^d and the k centers are points and the sum of squared distances of the lines to their closest of the k points points are to be minimized. The problem is said to have \"received little attention in existing literature\".\n\nI find The motivation of the model is vague, and it seems the reverse case has received a lot more attention, where the input consists of (possibly noisy) points from a linear flat and the centers are linear flats trying to reconstruct the true subspaces in which the points lie. Also when considering lines as inputs, why would they extend infinitely instead of just line segments, which would be a special case of polygonal curves clustering?\n\nThe problem in extending existing LS algorithms (for k-means) to the lines setting is of technical nature. For instance the tringle inequality does not hold and this complicates tracking the cost changes when reassigning from approximate centers to optimal centers. To this end a novel technique called \"proportional capture relation\" is introduced that also accounts for the number of reassigned lines to avoid large cost increase.\n\nThe second technical problem is in finding candidate swap pairs which is solved by a centroid set construction from previous coreset work [22] and a grid construction around the point where a pair of lines crosses (or attains its smallest distance in high dimensions)\n\nAlthough everything is claimed to be polynomial time in several places even in the high dimensional setting, I found C^d dependences which are clearly exponential (for the Cross Line grid construction).\n\nOnce those technicalities are solved, the analysis follows pretty much verbatim to recent work [18] on local search for the standard k-means of points, and more classic work (I am not sure but the right reference that is missing might be Vijay Arya, Naveen Garg, Rohit Khandekar, Adam Meyerson, Kamesh Munagala, and Vinayaka Pandit. Local search heuristics for k-median and facility\nlocation problems. SIAM Journal on Computing, 33(3):544–562, 2004. )\n\nSTRENGTHS AND WEAKNESSES:\n*Strength*\n- local search helps in the new setting\n- simple algorithm\n\n*Weakness*\n- weak motivation\n- high dimension dependence (C^d)\n- bad approximation factor 500 in expectation (so 1500 with probability 2/3 ?)\n- analysis very close to existing local search literature\n- lots of typos\n\nQUESTIONS:\n- perhaps the authors could comment on the motivation of clustering infinite lines with points, instead of the more established settings of (finite) line segments as in polygonal curve clustering, or clustering points with linear flats as in subspace clustering?\n\n- I would recommend to specify \"with polynomial time\" in Theorem 1\n\n- I find it confusing to read the \\delta(p,q) notation for the **squared** Euclidean distances. why not \\delta(p,q)^2 ? Or better |p-q|_2^2!!\n\n- in 3.1 could you explain why is the choice of the smallest distance between two lines good?\n\n- below Lemma 3 two mentions of [22] where it should be [18] ?!\n\n- definnition 6 and running time analysis: 9^d does not look very polynomial to me ?!\n\nLIMITATIONS:\nLimitatios section is very limited. I think noise-freeness is minor compared to the *real* limitations such as exponential d dependence, very high approximation factor etc...", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}