How the result of graph clustering methods depends on the construction of the graph

Maier, Markus; von Luxburg, Ulrike; Hein, Matthias

ESAIM: Probability and Statistics, Tome 17 (2013), pp. 370-418.

We study the scenario of graph-based clustering algorithms such as spectral clustering. Given a set of data points, one first has to construct a graph on the data points and then apply a graph clustering algorithm to find a suitable partition of the graph. Our main question is if and how the construction of the graph (choice of the graph, choice of parameters, choice of weights) influences the outcome of the final clustering result. To this end we study the convergence of cluster quality measures such as the normalized cut or the Cheeger cut on various kinds of random geometric graphs as the sample size tends to infinity. It turns out that the limit values of the same objective function are systematically different on different types of graphs. This implies that clustering results systematically depend on the graph and can be very different for different types of graph. We provide examples to illustrate the implications on spectral clustering.

[1] D. Angluin and L. Valiant, Fast probabilistic algorithms for Hamiltonian circuits. J. Comput. Syst. Sci. 18 (1979) 155-193. | MR | Zbl

[2] G. Biau, B. Cadre and B. Pelletier, A graph-based estimator of the number of clusters. ESAIM: PS 11 (2007) 272-280. | Numdam | MR | Zbl

[3] M. Brito, E. Chavez, A. Quiroz and J. Yukich, Connectivity of the mutual k-nearest-neighbor graph in clustering and outlier detection. Stat. Probab. Lett. 35 (1997) 33-42. | MR | Zbl

[4] S. Bubeck and U. Von Luxburg, Nearest neighbor clustering: a baseline method for consistent clustering with arbitrary objective functions. J. Mach. Learn. Res. 10 (2009) 657-698. | MR | Zbl

[5] J.W. Harris and H. Stocker, Handbook of Mathematics and Computational Science. Springer (1998). | MR | Zbl

[6] W. Hoeffding, Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 58 (1963) 13-30. | MR | Zbl

[7] D.O. Loftsgaarden and C.P. Quesenberry, A nonparametric estimate of a multivariate density function. Ann. Math. Stat. 36 (1965) 1049-1051. | MR | Zbl

[8] M. Maier, M. Hein and U. Von Luxburg, Optimal construction of k-nearest neighbor graphs for identifying noisy clusters. Theoret. Comput. Sci. 410 (2009) 1749-1764. | MR | Zbl

[9] M. Maier, U. Von Luxburg and M. Hein, Influence of graph construction on graph-based clustering measures, in Advances in Neural Information Processing Systems, vol. 21, edited by D. Koller, D. Schuurmans, Y. Bengio and L. Bottou. MIT Press (2009) 1025-1032.

[10] G. Miller, S. Teng, W. Thurston and S. Vavasis, Separators for sphere-packings and nearest neighbor graphs. J. ACM 44 (1997) 1-29. | MR | Zbl

[11] H. Narayanan, M. Belkin and P. Niyogi, On the relation between low density separation, spectral clustering and graph cuts, in Advances in Neural Information Processing Systems, vol. 19, edited by B. Schölkopf, J. Platt and T. Hoffman. MIT Press (2007) 1025-1032.

[12] A. Srivastav and P. Stangier, Algorithmic Chernoff-Hoeffding inequalities in integer programming. Random Struct. Algorithms 8 (1996) 27-58. | MR | Zbl

[13] U. Von Luxburg, A tutorial on spectral clustering. Stat. Comput. 17 (2007) 395-416. | MR

[14] U. Von Luxburg, M. Belkin and O. Bousquet, Consistency of spectral clustering. Ann. Stat. 36 (2008) 555-586. | MR | Zbl