Lecture slides and references for CS240B (Tentative)

Notice that Section "READ" gives a weekly reading list and Section "Additional information" lists some helpful reference information.

• Week 1 (9/30, Oct 5). Introduction to Parallel Computing. Communication/computation model. SPMD. MPI Programming.

  Read.

  • Lecture notes on parallel scientific computing by Yang (Sections 1, 2 and appendix A).
  • A User's Guide to MPI. Click here to get the postscript file with 51 pages or the postscript file with 26 pages (2 in one page).

  Slides: Introduction and MPI programming.
  

  Additional information:

  • Chapter 1 of Culler/Singh/Gupta's book (CSG) "Parallel Computer Architecture: A Hardware/Software approach" published by Morgan Kuffman (1998) PDF and PS file is available here.
  • MPI Parallel Programming on SGI Origin 2000 at UCSB. HTML.
  • MPI Programming on CS PC machines. Text.
  • Sample code for MPI .
    
  • The Message Passing Interface (MPI standard 1.1), FUNCTION INDEX. HTML Or Another MPI site
  .

• Week 2 (Oct 7, Oct 12). Program transformation. Pthreads.

  Read.

  • Lecture notes on parallel scientific computing by Yang (Section 5 and appendix B).

  Slides: Parallelization issues (program transformation). and Pthread programming.
  

  Additional information:
  Sample code for pthreads .
  More pthreads info.
  

• Weeks 3 (Oct 14, Oct. 19/21). Parallel algorithms (matrix computation). Memory hierarchies and BLAS.

  Read. Lecture notes on parallel scientific computing by Yang (Sections 6-9 and appendix B).

  Slides: Parallel matrix algorithms. and Memory hierarchies and BLAS.

  Additional information:
  Netlib BLAS information
  

• Week 4: Shared memory machines (Oct 26, Oct 28, Nov 2, Nov 4) .

  Read. Chapters 5 and 8 of Culler/Singh/Gupta's book (CSG) "Parallel Computer Architecture: A Hardware/Software approach" published by Morgan Kuffman (1998).

  Slides: Chapter 5 (4x1 in ps format) only need slides 1-44 and 59-92
  Chapter 8 (4x1 in ps format) only need page 1-58

  Additional info: All CSG slides in different formats

• Week 5 (Nov 4, Nov 9): Systems support in shared-memory machines .
  • Pthreads for Dynamic and Irregular Parallelism by Girija J. Narlikar, Guy E. Blelloch, In SC'98 (High Performance Networking and Computing) Paper (compressed postscript). Slides (postscript). Click here to get the more info.
    
  • H. Tang, K. Shen, T. Yang. Compile/Run-time Support for Threaded MPI Execution on Multiprogrammed Shared Memory Machines, in PPoPP'99. Paper (postscript). Slides .
    Shen, H. Tang, T. Yang, Adaptive Two-level Thread Management for Fast MPI Execution on Shared Memory Machines, in SC'99. Click here to get more info
    

  Additional information.

  • A Comparison of the MPI, SHMEM and Cache-coherent Shared Address Space Programming Models on the SGI Origin2000. H. Shan and J. P. Singh. Proc. Intl. Conference on Supercomputing, June 1999. Click here to get papers/slides

• Weeks 6/7 (Nov 11, Nov 18): Cluster computing systems and applications .

  Papers and slides.

  • Slides by A. Chien on Design and Expereince with Supercomputing Clusters of PCs (PS file).
    Paper: "Design and Evaluation of an HPVM-based Windows NT Supercomputer". To appear in Parallel Processing Letters and Internal Journal of High-Performance and Scientific Applications (By A. Chien, et al) Click here to get more info
  • A. C. Arpaci-Dusseau, D. E. Culler, A. Mainwaring. Scheduling with Implicit Information in Distributed Systems in Sigmetrics'98. Paper (compressed postscript). Slides (compressed postscript). Click here to get the more info.
  • H. Zhu, B. Smith and T. Yang. Scheduling Optimization for Resource-Intensive Web Requests on Server Clusters, in SPAA'99. Click here to get the paper and slides.
    
  • Locality-Aware Request Distribution in Cluster-Based Network Servers V. Pai, M. Aron, G. Banga, M. Svendsen, P. Druschel, W. Zwaenepoel, E. Nahum, ASPLOS'98 Paper (compressed postscript). Slides (compressed postscript). Click here to get more info

• Week 8 (Nov 23, Nov 30): Parallel I/O. Wide-area computing.

  Slides:

  • MPI-IO slides by Thakur (gziped PS file) .
  • The Globus project (32 pages).

  Additional information

  • The Globus Project: A Status Report. I. Foster, C. Kesselman, Proc. IPPS/SPDP '98  Heterogeneous Computing Workshop, pg. 4-18, 1998. Paper (compressed ps file). Click here to get more info.
  • MPI-2 (I/O)
  • R. Thakur, W. Gropp, and E. Lusk, ``On Implementing MPI-IO Portably and with High Performance,'' in Proc. of the Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999, pp. 23--32. click here to get the paper

• Week 9 (Dec 2, 7): Parallelizing Compilers and OpenMP.

  Slides:

  • Slides on parallelizing compilers.
    
  • OpenMP slides [PostScript] [PDF]. (Postscript slides may not be viewed correctly. Printing is okay)
    

  Additional information:

  • OpenMP: An Industry-Standard API for Shared-Memory Programming in IEEE Computational Science & Engineering, Vol. 5, No. 1, January/March 1998
    
  • Compiler transformations for high-performance computing in ACM Computing Survey, 1994.
  • OpenMP homepage. It contains OpenMP spec for C++ and Fortran, samples, tutorials and software.
  • The SUIF parallelizing compiler software.