CS 290H, Spring 2014: References

This list is still under construction!

Reference to notation and definitions

• CS 290H index. A numbered list of definitions, notation, and key theorems for the course. There is a lot of variation in terminology and notation in the literature on spectral graph theory; this list gives the version we will use. The list will grow during the quarter.

• See especially Dan Spielman's online lecture notes. I will be assigning readings from Spielman's notes, among other papers.

• Algorithmic spectral graph theory boot camp. Videos of a one-week program of introductory lectures from a 2014 workshop at the Simons Institute at Berkeley.

• Fan Chung. Spectral Graph Theory. AMS, 1997. Here are some recently updated chapters. This is a wonderfully thorough and complete book on the mathematics, but not the computational aspects, of graph spectra. Warning: The definitions in this business vary from one place to another, and what Chung calls the "Laplacian" is what we (and Spielman) call the "normalized Laplacian". They are closely related but have different eigenvalues and eigenvectors in general.

• Chris Godsil and Gordon Royle. Algebraic Graph Theory. Springer, 2001. Very comprehensive background on graph theory and its algebraic connections, including the spectra of Laplacians and other graph-based matrices. Chapter 1 is a good quick overview of graph definitions. Chapter 13 is a good, fairly brief, overview of graph Laplacians. Godsil and Royle's Laplacian is the same as ours and Spielman's, though they give a different definition in terms of incidence matrices.

Applications in machine learning, statistical inference, and data analysis

• Aaron Clauset's introductory lecture on stochastic blockmodels. A widely studied generative graph model for clustering behavior.

• D. L. Sussman, M. Tang, D. E. Fishkind, and C. E. Priebe. A consistent adjacency spectral embedding for stochastic blockmodel graphs. JASA 107: 1119-1128, 2012. Carey writes: See Figure 3.

• P. Sarkar and P. J. Bickel. Role of normalization in spectral clustering for stochastic blockmodels. Dan Sussman writes: The recent paper that we really like regarding the A vs L issue. Figure 1 shows the asymptotic ratio between the "variance" of the eigenvectors of the (normalized) Laplacian vs the "variance" of the eigenvectors of the adjacency. It shows that for many parameter settings the (normalized) Laplacian is to be preferred, at least when using k-means for subsequent clustering.

• T. Qin and K. Rohe. Regularized spectral clustering under the degree-corrected stochastic blockmodel. NIPS 2013. Dan Sussman writes: Another paper, which is also interesting, studies another Laplacian type object and has results showing that very sparse graphs can still be clustered using spectral methods on this matrix. It's quite simple as well and I wonder if its nice theoretical properties also translate into nice computational properties.

• C. Seshadhri, Tamara G. Kolda, and Ali Pinar. Community structure and scale-free collections of Erdős-Rényi graphs. A generative graph model called BTER that's sort of related to stochastic block models, but tries to match degree distributions and local clustering behavior explicitly.

• Tamara G. Kolda, Ali Pinar, Todd Plantenga, and C. Seshadhri. A scalable generative graph model with community structure. Algorithmic considerations and experimental results with BTER.

• Lin and Cohen. Power iteration clustering. A spectral clustering method that tries to use one vector to find several clusters.

• U. von Luxburg. A tutorial on spectral clustering.

Applications in graph algorithms, partitioning, network flow, etc.

• F. McSherry. Spectral partitioning of random graphs. FOCS 2001. A classic paper showing that Laplacian methods can find good partitions, colorings, and cliques in most graphs that have them.

• A. Pothen, H. D. Simon, and K.-P. Liou. Partitioning sparse matrices with eigenvectors of graphs. SIAM. J. Matrix Anal. Appl. 11: 430–452, 1989. A classic paper that introduced the spectral partitioning heuristic to the scientific computing community.

• T. F. Chan, J. R. Gilbert, and S. H. Teng. Geometric spectral partitioning. Xerox PARC technical report, 1995.

• Dan Spielman and Shanghua Teng. Spectral partitioning works: Planar graphs and finite element meshes. Proc. 37th FOCS, IEEE, 1996. Good separators for planar graphs by repeatedly using low-conductance spectral cuts.

• Jon Kelner. Spectral partitioning, eigenvalue bounds, and circle packings for graphs of bounded genus. SIAM Journal on Computing 35.4: 882-902, 2006. Extends the Spielman/Teng spectral proof the planar separator theorem to graphs of genus g.

• P. Christiano, J. A. Kelner, A. Madry, D. A. Spielman, and S.-H. Teng. Electrical flows, Laplacian systems, and faster approximation of maximum flow in undirected graphs. Proc. 43rd STOC, ACM, 2011. Finding a max flow by solving a sequence of Laplacian linear systems.

• J. A. Kelner, G. L. Miller, and R. Peng. Faster approximate multicommodity flow using quadratically coupled flows. Proc. 44th STOC, ACM, 2012.

• Y. T. Lee, S. Rao, and N. Srivastava. A new approach to computing maximum flows using electrical flows. Proc. 45th STOC, ACM, 2013.

Applications of nonsymmetric matrices of graphs

• F. Krzakala, C. Moore, E. Mossel, J. Neeman, A. Sly, L. Zdeborová, and P. Zhang. Spectral redemption: clustering sparse networks.

• Q. Zhu, G. Song, and J. Shi. Untangling cycles for contour grouping.

Algorithms to compute eigenvectors and eigenvalues

• Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. Van der Vorst, editors. Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide. Available free online or in hardcopy from SIAM. This is a comprehensive and invaluable book on all kinds of eigenvalue and generalized eigenvalue problems, both Hermitian (the kind we're interested in) and not.

• Jim Demmel. Applied Numerical Linear Algebra. SIAM, 1997. This is a wonderful book and you should buy it if you plan to do anything in computational science or numerical analysis. It has excellent chapters on conjugate gradients and Lanczos.

• R. Grimes, J. Lewis, and H. Simon. A shifted block Lanczos algorithm for solving sparse symmetric generalized eigenproblems. A paper about an industrial-strength eigensolver that deals with many of the tricky computational and numerical issues that arise in practice.

• L. N. Trefethen and D. Bau. Numerical Linear Algebra. This is the most readable and elegant introduction to numerical methods for eigenanalysis and linear system solving, emphasizing iterative methods.

• Jonathan Shewchuk, An introduction to the conjugate gradient method without the agonizing pain. What it says. A classic.

• R. Barrett and nine co-authors, Templates for the Solution of Linear Systems. A terse user's guide to iterative methods for solving linear systems. Code in C, C++, Fortran, and Matlab is available as well.

• Dan Spielman, Algorithms, graph theory, and linear equations in Laplacian matrices. A very readable survey.

• Nisheeth Vishnoi, Lx = b: Laplacian solvers and their algorithmic applications. A long, detailed, valuable survey paper from a theoretical point of view.

• Sivan Toledo and Haim Avron's book chapter on combinatorial preconditioning.

• Doron Chen and Sivan Toledo's experiments with the original Vaidya support graph preconditioners.

• Erik Boman and Bruce Hendrickson, Support theory for preconditioning. A very clean linear algebraic framework for combinatorial preconditioning.

• Erik Boman, Doron Chen, Bruce Hendrickson, and Sivan Toledo, Maximum-weight-basis preconditioners. Extends support theory to factor-width-2 (i.e. symmetric diagonally dominant) matrices.

• Dan Spielman and Shang-Hua Teng, Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. An elaborate and beautiful theoretical result; the work of teasing out a practical method is still going on.

• Ioannis Koutis, Gary L. Miller and Richard Peng, Approaching optimality for solving SDD systems. A large step toward making the Spielman-Teng approach practical.

• Koutis, Miller, and Tolliver, Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing. Implementation and experiments with a combinatorial multigrid method (which they call CMG) that uses some of the ideas in Spielman/Teng and Koutis/Miller/Peng.

• Jon Kelner, Lorenzo Orecchia, Aaron Sidford, and Allen Zhu. A simple, combinatorial algorithm for solving SDD systems in nearly-linear time. A strikingly simple method (not preconditioned CG) that uses some ideas from support theory.

• Oren Livne and Achi Brandt. Lean algebraic multigrid (LAMG): Fast graph Laplacian linear solver. Using multigrid ideas for Laplacian linear systems.

• Seymour Parter. The use of linear graphs in Gauss elimination. The classic 1961 paper that first modeled Gaussian elimination as a process on graphs, and proved that trees can be factored by GE without fill.

Software

• CSparse, a comprehensive library of C sparse matrix routines by Tim Davis that can be used either standalone or with Matlab.

• TAUCS, a library of software by Sivan Toledo's group at Tel-Aviv that contains direct solvers and many different preconditioners, including several support-graph preconditioners.

• MatlabBGL, a package of Matlab graph routines by David Gleich that uses the Boost Graph Library.

• Meshpart, a somewhat older package of Matlab graph routines that I tend to use for demos because I know it pretty well :).

• BTER and Feastpack, a library by Kolda, Pinar, et al. that includes a good generator for random graphs with specified degree distribution and clustering behavior.

Related courses elsewhere and elsewhen

• Dan Spielman's course at Yale: "Spectral Graph Theory". This is the pioneering course on which I'm basing the theoretical parts of CS 290H. Dan's online lecture notes are terrific.

• Gary Miller's course at Carnegie-Mellon: "Spectral Graph Theory and the Laplacian Paradigm". A great mixture of theory and practice, aimed at goals similar to ours.

• Luca Trevisan's course at Stanford: "Graph Partitioning and Expanders". This course covers several applications of spectral graph theory from a very theoretical (i.e. STOC/FOCS conference) point of view. Luca's online lecture notes are excellent.

• Simons Institute fall 2014 program on "Algorithmic Spectral Graph Theory", including several workshops on topics related to CS290H.

• Aaron Clauset's course at Colorado: "Network Analysis and Modeling". This course isn't about graph Laplacians, but it has lots of good material (and lecture notes) about using graphs to model and analyze social and biological phenomena.