CS240A: Applied Parallel Computing: Spring 2007

Interdisciplinary introduction to applied parallel computing on modern supercomputers. Topics include applications oriented architecture, software systems, MPI, UPC, parallel MATLAB, caches and vector processors, and parallel algorithms for algorithms such as linear algebra, N-body problems, multigrid, fast multipole, mesh partitioning, and graph algorithms. The course includes a project with hands-on emphasis on understanding the realities and myths of what is possible on the world's fastest machines.

Class Meeting

Mondays and Wednesdays 09:00-10:30, Trailer 932.

Instructor

• Office: Phelps 3220
• Office hours: see my web page.
• Email: gilbert@cs.ucsb.edu

Assistant

• Email: kyolee_at_cs_dot_ucsb_dot_edu
• Office hours: Wednesdays 1:00 - 3:00 in GSL (trailer near library)

Discussion and Announcements

There is a Google discussion group for the class at http://groups.google.com/group/UCSB-Computer-Science-240A-Spring-2007. All students are expected to join this group and to watch it for course announcements. You may also use it to post questions (of general interest to the class) for the instructor and assistant. The mailing address for posting to the group is UCSB-Computer-Science-240A-Spring-2007@googlegroups.com.

Language and Machine Documentation

• Stefan Boeriu's DataStar information page (includes lots of info on MPI performance and trace tools, such as VAMPIR)
• Kyo Lee's page of DataStar hints and examples

• Peter Pacheco's introduction to MPI
• An extensive tutorial on MPI from Livermore Labs, with lots of examples.
• Documentation for all the MPI routines.
• Examples of MPI code

• UPC Manual
• The official UPC web site
• A tutorial on UPC (and also Co-array Fortran and Titanium, which are similar PGAS languages based on Fortran and Java respectively).
• UPC performance tools: PPW, UPC_trace
• Examples of UPC code

Course Requirements

• Research homework 0: 10%
• Programming homeworks (3): 45%
• Final project: 40%
• Class participation: 5%

Homework

• Account Setup Instructions for Homeworks (Read this first!): Instructions
• Homework 0 (due Wed Apr 11): Problem, Results
• Homework 1 (due Wed Apr 18): Problem, FAQ, Sequential Matlab code, Solutions
• Homework 2 (due Wed May 2): Problem, FAQ, Sequential Matlab code
• Homework 3 (due Wed May 16): Problem, FAQ, Sequential Matlab code

Final Project

• Project web pages
• Schedule for final presentations and demos
• Due dates and requirements
• Last year's project web pages
• Year before last's project web pages
• Some project ideas

Class Schedule and Lecture Slides

• Mon Apr 2: Introduction [bone density simulation, top 500 list, DataStar, Columbia and Beowulf, XMT (Eldorado), Cell]

• Wed Apr 4: Models of parallel programming and architecture [slides]

• Mon Apr 9: Parallel programming language intro: MPI and UPC [slides, MPI examples, UPC examples]

• Wed Apr 11: More on UPC and the PGAS model [hpcs study]
  (Homework 0 due)

• Mon Apr 16: High-performance linear algebra libraries (Julie Langou, UTK) [slides]

• Wed Apr 18: Memory locality and matrix multiplication [lecture slides, HPCS defect study slides, HPC BugBase website]
  (Homework 1 due)

• Mon Apr 23: Two multicore architectures: Opteron general-purpose (Keshava Subramanya); NVidia GPU (Brent Oster) [slides]

• Wed Apr 25: Parallel matrix multiplication [same slides as Apr 18]

• Mon Apr 30: Parallelism in physical simulation I [lecture slides, Sharks and Fish]

• Wed May 2: Parallelism in physical simulation II [same slides as Apr 30]
  (Homework 2 due)

• Mon May 7: Partial differential equations and sparse linear systems [my slides, temperature demo files]

• Wed May 9: Solving linear systems: The conjugate gradient method [same slides as May 7, airfoil demo files]
  (Final project teams and proposals due)

• Mon May 14: Solving linear systems: Dense and sparse Gaussian elimination [same slides as May 7, Jim Demmel on dense solvers, Sherry Li on sparse solvers, airfoil demo files]

• Wed May 16: Graph partitioning [my slides on theory and Fiduccia-Mattheyses, Alan Edelman's slides on spectral and geometric methods, Kathy Yelick's overview slides]
  (Homework 3 due)

• Mon May 21: Parallel graph algorithms [slides; papers on maximal independent sets, connected components, and strongly connected components; sequential and parallel MIS programs]

• Wed May 23: Trees, scans, PRAMs, and log n [slides, Guy Blelloch on scans]
  (Final project progress reports due)

• Mon May 28: (Memorial Day, no class.)

• Wed May 30: Matlab*P (Viral Shah)

• Mon Jun 4: Final project presentations [schedule]

• Wed Jun 6: Final project presentations [schedule]

• Fri Jun 8: Final project demos, 9:00 am to noon in GSL [schedule]

• Mon Jun 11: (Final project papers due)

Interesting Links

• Web site and white paper from Berkeley on the future of parallel computing in the multicore era.
• Top 500 SuperComputers
• Top 10 Algorithms
• Dead SuperComputer Society
• Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers
• A nice introduction to finite element methods in a biomedical application
• Horst Simon's Feb 2005 CSE panel talk

Similar Courses Elsewhere and Elsewhen

• Applied Parallel Computing, MIT (link)
• Applications of Parallel Computers, UC Berkeley (link)
• CS240A, Winter 2006, UCSB (link)