CS140: Parallel Scientific Computing: Winter 2007

Introduction to the theory and practice of parallel processing and high-performance computer architecture. Topics include: parallel algorithm design for scientific computation; message-passing parallel programming using MPI; parallel performance evaluation and tuning; parallel numerical algorithms and applications. Students will gain experience with programming some of the world's fastest machines.

Instructor

• Email: gilbert@cs.ucsb.edu
  
• Office: Phelps 3220
  
• Office hours: see my web page.

Teaching Assistant

• Email: veljkoveljko@gmail.com
  
• Phone:
  
• Office hours: Fridays 12:00 - 3:00 in CSIL
  

Discussion and Announcements

There is a Google discussion group for the class at http://groups.google.com/group/ucsb-computer-science-140-winter-2007. All students are expected to join this group and to watch it for course announcements. You may also use it to post questions for the instructor and assistant (and answers to other people's questions). The mailing address for posting to the group is ucsb-computer-science-140-winter-2007@googlegroups.com.

Class Meetings

• Lecture: Monday and Wednesday 3:30-4:45, North Hall 1111
  
• Discussion: Friday 11:00-11:50, Phelps 3519

Textbook

The textbook is: Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers (second edition, 2005), by Barry Wilkinson and Michael Allen. The book has an online home page with sample programs.

I will assign reading from the textbook each week; see the daily schedule below. You will need to both read the text and come to class. You are responsible for material we cover in class whether it is in the textbook or not (quite a bit of it won't be). Class participation is expected and will have an influence on your grade.

Computer Resources

You will have two parallel computers to use for programming assignments and projects. Please make sure you get accounts on both of them.
• Our local Computer Science / IGERT cluster. Your account on the cluster will be created automatically if you were preregistered for CS140 by January 4, and you already have an Engineering College (CSIL) account. If not, you must first sign up for a CSIL account at https://accounts.engr.ucsb.edu/create_account.php, specifying that you're in CS140 and giving your perm number; then you must send email to support@cs.ucsb.edu asking them to give your CSIL account access to the cluster. The CS140 section on Friday, January 12 will be a tutorial on how to run programs on the cluster.

• The DataStar machine at the San Diego Supercomputer Center. Your account on DataStar will be created automatically if you were preregistered for CS140 by January 4. If not, you must send me email immediately with your name and perm number. It takes a week or so to set up DataStar accounts, so please do this right away. In class Wednesday, January 17, Stefan Boeriu from ECI will give an overview and tutorial on using DataStar.

MPI Resources

You will write programs in C using MPI, which is a library of routines for message-passing parallel programming that is implemented on almost all parallel systems. Here are some places to look for useful information on MPI. You can find more by Googling around on the web.

• The textbook has an introduction to MPI in Sections 2.1 and 2.2, and its Appendix A describes 19 of the MPI routines. MPI has hundreds of routines in all, but these 19 will be more than you need for all the programming you'll do in CS 140.

• The examples from class Jan 10 and from section Jan 12.

• A tutorial set of slides by Bill Gropp at Argonne Labs.

• A nice gentle but thorough introduction to MPI by Peter Pacheco.

• A short introduction to MPI, working up to an example of a Poisson solver, by Jimack and Touheed at Leeds.

• Norm Matloff's MPI page at UC Davis has lots of good pointers to information and examples.

• An Argonne Labs list of tutorials

• The Argonne Labs man pages for all the MPI routines.

• An extensive tutorial on MPI from Livermore Labs, with lots of examples and an appendix with documentation for all the MPI routines.

Grading

• Homework assignments: 35%
• Midterm exams (2): 30%
• Final project: 35%

Homework Policy

There will be a number (probably 3) of homework assignments, and a final project. The goal of the final project is to demonstrate your ability to design and implement a parallel program to solve a problem in scientific computing. For the homework assignments and final project, you may work alone or in groups of two. You will write a report describing your approach to the final project and presenting your results.

It is your responsibility to manage your time so that you can turn in your assignments when they are due. No late homework will be accepted.

If you have questions about grading of homework, talk to the t.a. first. If you are unable to reach an agreement, make an appointment to talk to me. The statute of limitations for regrades is one week -- that is, any requests for regrades must be made no later than one week after the homework (or exam) was returned in class.

Homework Assignments

• Homework 0, due Tuesday, January 23.
• Homework 1, due Wednesday, January 31.
• Homework 2, due Friday, February 16. [SUMMA slides (see slides 31-35), SUMMA paper, Pacheco MPI tutorial, DataStar ESSL and PESSL libraries, Hadamard matrices]

Exams

• Midterm 1: Wednesday, February 7, in class. [rules and syllabus]
  Open textbook and notes; no other books or computers; no sharing books.

• Midterm 2: Wednesday, March 7, in class. [rules and syllabus]
  Open textbook and notes; no other books or computers; no sharing books.

Final Project

For the final project, you will choose one of the three problems described at the link below, write a project proposal in which you describe in detail what you plan to do, and then do an implementation and experiment following your proposal.

The projects are intentionally left open-ended so that you can explore. I encourage you to discuss your plans with me and/or Veljko, in office hours or by email, as you decide what to do. The project proposal is intended as a chance for you to get feedback on your plans before you get too far down the road, so please make your proposal as complete as possible even if you're not yet sure what might change.

I recommend doing the term project in teams of two, though you may do it alone if you prefer.

• Link to Final Project Problem Descriptions.

• Friday, March 2: Project teams and proposals due. Turn in a one- or two-page report giving the names of the students on your team, saying which project you are going to do, and then going into some detail about what algorithms you will implement, what experiments you will do, and what results you will present.

• Monday, March 12: Progress reports due. Turn in a one- or two-page report giving the status of your project, what is going well and what is giving you difficulty, and any changes you want to make to your proposal.

• Wednesday, March 21: Final project reports due. Turn in a report of about 5 pages describing in detail the algorithms you used and the experiments you did, and your interpretation of the results. What turned out as you expected? Did anything about your results surprise you? If so, can you explain it? Use both tables and graphs to show your results. Also turn in the code for your project program.

Class Schedule, Links, Slides, and Reading Assignments

• Mon Jan 8: Introduction. [bone density simulation, Columbia and Beowulf, top 500 list, DataStar, XMT (Eldorado), Cell]
  Reading: Textbook sections 1.1, 1.2, 1.3 (overview of parallel computers).

• Wed Jan 10: Message-passing and MPI. [slides, examples]
  Reading: Textbook sections 2.1, 2.2 (intro to MPI).

• Mon Jan 15: (Martin Luther King Day, no class.)

• Wed Jan 17: DataStar overview, accounts, and tools. Guest speaker: Stefan Boeriu, UCSB ECI. [Stefan's web page]
  (Homework 0 assigned)

• Mon Jan 22: Measures of parallel complexity. Embarrassingly parallel problems: Monte Carlo methods.
  Reading: Textbook sections 2.3, 2.4 (evaluating parallel programs) and sections 3.1 - 3.3 (embarrassingly parallel computation).
  (Homework 1 assigned)

  (Homework 0 due 11:59pm Tue Jan 23)

• Wed Jan 24: Tightly coupled problems: communication measures, matrix-vector multiplication.
  Reading: Textbook sections 4.1.1 (partitioning), 11.1, 11.2 (matrix multiplication).

• Mon Jan 29: More matrix algorithms.

• Wed Jan 31: Matrix-matrix multiplication. [slides]
  (Homework 2 assigned)

  (Homework 1 due 11:59pm Wed Jan 31)

• Mon Feb 5: Spatially local problems: Stencils, finite differences, the game of Life.
  Reading: Textbook sections 6.1 - 6.3 (synchronous iteration).

• Wed Feb 7: Midterm 1. [rules and syllabus]

• Mon Feb 12: Parallel programming in Matlab. Guest speaker: Viral Shah.

• Wed Feb 14: Details of SUMMA. Guest speaker: Viral Shah.

  (Homework 2 due 11:59pm Fri Feb 16)

• Mon Feb 19: (Presidents' Day, no class.)

• Wed Feb 21: Mesh computations, domain decomposition, Jacobi relaxation.
  Reading: Textbook sections 6.1 - 6.3 (synchronous iteration), 11.3 - 11.4 (linear equations and iterative methods).

• Mon Feb 26: Guest speaker: Peter Fritzson, CS Colloquium on Parallel and Grid Modeling (note different time and place: 3:00 - 4:00, ESB 2100).

• Wed Feb 28: (JRG travel, no class.)

  (Final project teams and proposals due 11:59pm Fri Mar 2)

• Mon Mar 5: Parallel stencil methods. [Jacobi iteration slides]

• Wed Mar 7: Midterm 2. [rules and syllabus]

• Mon Mar 12: Sparse matrix/vector multiplication, matrices and graphs. [slides]

  (Final project progress reports due 11:59pm Mon Mar 12)

• Wed Mar 14: Graph algorithms. [Paritioning slides: Kathy Yelick on overview and multilevel, Alan Edelman on spectral and geometric, Fiduccia/Mattheyses example; Luby's parallel independent set algorithm; sequential and parallel MIS programs]

  (Final projects due 11:59pm Wed Mar 21)