CS140: Parallel Scientific Computing: Winter 2008

Introduction to the theory and practice of parallel processing and high-performance computer architecture. Topics include: parallel algorithm design for scientific computation; message-passing parallel programming using MPI; parallel performance evaluation and tuning; parallel numerical algorithms and applications. Students will gain experience with programming some of the world's fastest machines.

Instructor

• Email: gilbert@cs.ucsb.edu
  
• Office: Harold Frank Hall 5109
  
• Office hours: see my web page.

Teaching Assistant

• Email: nlarusso@cs.ucsb.edu
  
• Phone:
  
• Office hours: Mondays 1pm - 3pm in CSIL
  

Discussion and Announcements

There is a Google discussion group for the class at http://groups.google.com/group/ucsb-computer-science-140-winter-2008. All students are expected to join this group and to watch it for course announcements. You may also use it to post questions for the instructor and assistant (and answers to other people's questions). The mailing address for posting to the group is ucsb-computer-science-140-winter-2008@googlegroups.com.

Class Meetings

• Lecture: Tuesday and Thursday 9:30-10:45, North Hall 1111
  
• Discussion: Friday 12:00-12:50, Girvetz 1112

Textbook

The textbook is: Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers (second edition, 2005), by Barry Wilkinson and Michael Allen. The book has an online home page with sample programs.

I will assign reading from the textbook each week; see the daily schedule below. You will need to both read the text and come to class. You are responsible for material we cover in class whether it is in the textbook or not (quite a bit of it won't be). Class participation is expected and will have an influence on your grade.

Computer Resources

• For programming homeworks and the final project, you will use the DataStar machine at the San Diego Supercomputer Center. Your account on DataStar will be created automatically if you were preregistered for CS140 by January 4. If not, you must send me email immediately with your name and perm number. It takes a week or so to set up DataStar accounts, so please do this right away. In class on Tuesday, January 15, Stefan Boeriu from ECI will give an overview and tutorial on using DataStar.

MPI Resources

You will write programs in C using MPI, which is a library of routines for message-passing parallel programming that is implemented on almost all parallel systems. Here are some good references on MPI. You can find more by Googling around on the web.

• The textbook has an introduction to MPI in Sections 2.1 and 2.2, and its Appendix A describes 19 of the MPI routines. MPI has hundreds of routines in all, but these 19 will be more than you need for all the programming you'll do in CS 140.

• Stefan Boeriu's DataStar information page (includes lots of info on MPI performance and trace tools, such as VAMPIR).
• Nick Larusso's page of DataStar hints and examples.

• User's Guide to MPI, by Peter Pacheco.
• Documentation for all the MPI routines.
• Tutorial on MPI from Livermore Labs, with lots of examples.
• Some examples of MPI code from class and section.
• Norm Matloff's MPI page at UC Davis has lots of sample programs and many other pointers.

Grading

• Homework assignments: 35%
• Midterm exams (2): 30%
• Final project: 35%

Homework Policy

There will be a number (probably 4) of homework assignments, and a final project. The goal of the final project is to demonstrate your ability to design and implement a parallel program to solve a problem in scientific computing. For the homework assignments and final project, you may work alone or in groups of two. You will write a report describing your approach to the final project and presenting your results.

It is your responsibility to manage your time so that you can turn in your assignments when they are due. No late homework will be accepted.

If you have questions about grading of homework, talk to the t.a. first. If you are unable to reach an agreement, make an appointment to talk to me. The statute of limitations for regrades is one week -- that is, any requests for regrades must be made no later than one week after the homework (or exam) was returned in class.

Homework Assignments

• Homework 1 (due Tue Jan 15): Problem, Results
• Homework 2 (due Thu Jan 24): Problem
• Homework 3 (due Thu Feb 7): Problem, Matlab code
• Homework 4 (due Sun Feb 24): Problem, SUMMA slides (see slides 31-35), SUMMA paper, Pacheco MPI tutorial, DataStar ESSL and PESSL libraries, Hadamard matrices

Exams

• Midterm 1: Tuesday February 12, in class. [rules and syllabus]
  Open textbook and notes; no other books or computers; no sharing books.

• Midterm 2: Tuesday March 11, in class.
  Open textbook and notes; no other books or computers; no sharing books. [rules and syllabus]

Final Project

For the final project, you will choose one of the three problems described at the link below, write a project proposal in which you describe in detail what you plan to do, and then do an implementation and experiment following your proposal.

The projects are intentionally left open-ended so that you can explore. I encourage you to discuss your plans with me and/or Nick, in office hours or by email, as you decide what to do. The project proposal is intended as a chance for you to get feedback on your plans before you get too far down the road, so please make your proposal as complete as possible even if you're not yet sure what might change.

If you have a parallel computing project of your own that you would like to work on instead of doing one of these three problems, please talk to us about it. It's okay to do a different project provided it has the right level of parallel-implementation difficulty, and provided you write a thorough proposal describing it.

I recommend doing the term project in teams of two, though you may do it alone if you prefer.

• Link to Final Project Problem Descriptions.

• Thursday, February 28: Project teams and proposals due. Turn in a one- or two-page report giving the names of the students on your team, saying which project you are going to do, and then going into some detail about what algorithms you will implement, what experiments you will do, and what results you will present.

• Thursday, March 6: Progress reports due. Turn in a one- or two-page report giving the status of your project, what is going well and what is giving you difficulty, and any changes you want to make to your proposal.

• Wednesday, March 19: Final project reports due. Turn in a report of about 5 pages describing in detail the algorithms you used and the experiments you did, and your interpretation of the results. What turned out as you expected? Did anything about your results surprise you? If so, can you explain it? Use both tables and graphs to show your results. Also turn in the code for your project program.

Class Schedule, Links, Slides, and Reading Assignments

• Tue Jan 8: Introduction. [bone density modeling, parallel sorting]
  Reading: Textbook sections 1.1, 1.2, 1.3 (overview of parallel computers).
  (Homework 1 assigned)

• Thu Jan 10: Message-passing and MPI. [slides on parallel machines]
  Reading: Textbook sections 2.1, 2.2 (introduction to MPI).

• Tue Jan 15: Stefan Boeriu: DataStar overview, accounts, and tools. [Stefan's DataStar page]
  (Homework 1 due)

• Thu Jan 17: More MPI; measures of parallel complexity. [MPI slides; examples of MPI code]
  Reading: Textbook sections 2.3, 2.4, 2.5 (analyzing and debugging parallel programs), and sections 3.1 and 3.2.3 (Monte Carlo methods).
  (Homework 2 assigned)

• Tue Jan 22: Parallel complexity and Amdahl's law. [DataStar architecture (slides 12-24)]
  Reading: Textbook sections 4.1.1 (partitioning), 11.1, 11.2 (matrix multiplication).

• Thu Jan 24: Tightly coupled problems: matrix-vector multiplication. [classnotes]
  (Homework 2 due, Homework 3 assigned)

• Tue Jan 29: Memory hierarchy and matrix multiplication. [slides]

• Thu Jan 31: Parallel matrix multiplication. [same slides as Jan 29]
  Reading: Textbook sections 6.1 - 6.3 (synchronous iteration).

• Tue Feb 5: Spatially local problems: Stencils, synchronous iteration, the game of Life. [Wikipedia on Life; a Java simulator]

• Thu Feb 7: More stencils, Jacobi relaxation. [slides]
  (Homework 3 due, Homework 4 assigned)

• Tue Feb 12: Midterm 1. [rules and syllabus]

• Thu Feb 14: Nick Larusso: MapReduce. [slides]

• Tue Feb 19: Load balancing in spatially local problems: Fish modeling. [paper]

• Thu Feb 21: Sparse matrices, graphs, partitioning. [slides on sparse matrices and partitioning]

  (Homework 4 due 11:59pm Sun Feb 24)

• Tue Feb 26: Viral Shah: Star-P.

• Thu Feb 28: Graph algorithms. [slides; sequential and parallel maximal independent set programs; Luby's paper on parallel MIS]
  Reading: "Graph Analysis with High-Performance Computing", passed out in class.

  (Final project teams and proposals due 11:59pm Thu Feb 28)

• Tue Mar 4: Graph algorithms and multithreaded architectures.

• Thu Mar 6: Brent Oster: GPU programming and CUDA.

  (Final project progress reports due 11:59pm Thu Mar 6)

• Tue Mar 11: Midterm 2. [rules and syllabus]

• Thu Mar 13: No class (JRG at SIAM Parallel Processing Conference)

• Wed Mar 19: Final project demos in CSIL, 8:00-11:00 am.

  (Final project reports due 11:59pm Thu Mar 20)