The main theme of this course will be software dependability. We will discuss formalisms, tools, and techniques for developing dependable software. The list of papers we will cover is given below. There will be several homeworks. Each student will do either a survey, a case study, or a research project relating to the topics discussed in the class.

Instructor: Tevfik Bultan (bultan@cs.ucsb.edu, Phone: x3735)
Office Hours: Monday-Wednesday 2:00PM-3:00PM at HFH 2123

TA: Puneet Lakhina (puneet@cs.ucsb.edu)
Office Hours: Tuesday-Thursday 10:00AM-11:00AM at GSL.

Class Webpage: URL: http://www.cs.ucsb.edu/~cs272/

Class Announcements Mailing List: https://lists.cs.ucsb.edu/mailman/listinfo/cs272

Lectures: Monday/Wednesday 11:00AM-12:50PM, 932 101

Presentation Schedule: The final presentation schedule is now available. Click here for the schedule

Course Work

• Reading Assignments: You have to read the papers that are listed under reading assignments.
• Homeworks: There will be several homeworks. Homeworks should be done individually. Please consider the homeworks as take home exams and do not discuss homework problems with other students. If homework problems are not clear ask for clarification in class or contact the TA or the instructor.
• Survey, Case Study, or Research Project: Each student is required to do one of the following: a survey, a case study or a research project. Surveys and case studies are done individually, whereas research projects can be done as teams (at most 3 students per team). Depending on the class size I plan to schedule presentations at the end of the class.
  • Survey: If you choose this option you are required to write a survey paper (about 10 pages long). To write a survey paper you have to choose a topic related to one of topics we will discuss in the class. You need to identify around 10 research papers on the topic you chose and read them. Your survey paper should explain the results in these papers and how they relate to each other.
  • Case Study: If you choose this option you are required to conduct a case study using one of the tools we discuss in the class or a related tool. You need to install the tool and find an application that it can be used for. You need to write a report (about 10 pages long) explaining the case study. Your report should explain the application, the results, and evaluate the tool you used.
  • Research Project: If you choose this option you are required to either (1) implement an extension to one of the tools we discuss in the class or a related tool, or (2) implement a tool related to one of the topics we discuss in the class. Some suggested project topics will be listed in the class webpage. Team projects are allowed (the teams can have at most three students). You need to write a report (about 10 pages long) explaining the tool you built. You need to do a demo at the end of the quarter explaining the tool you built.
• Progress Report: You are required to write a 3-5 page progress report about your survey, case-study, or research project. In the progress report you should explain the topic of your study, and discuss your findings so far. The progress reports will be due November 5th.

Choosing survey/case-study/research project topic

You need to send me an e-mail (before October 8th) about your survey/case-study/research project topic. You can choose any tool or topic listed below or you can choose a tool or a topic related to this course.

For survey: Send me a paragraph describing the topic and at least 3 references as a starting point.

For case-study: Send me a paragraph identifying the tool you are going to use in your case study and a description of what you are going to use it for. If too many students request to use the same tool I may ask you to change your selection. This will be first come first serve.

For research project: You need to send me a project proposal and names of the team members, and you should come to my office hours and talk to me about the project.

Some research project topics are available here

Project list is available here

Project Assignments

• Progress Report: Please prepare a 3-5 page progress report about your survey, case-study, or research project. In the progress report you should explain the topic of your study, and discuss your findings so far. Please turn in your progress reports on November 5th by 5:00PM.
• Final Report: The final reports (about 10 pages long) for each survey, case-study, or research project are due on Monday, December 8th by 5:00PM.
• Presentation: There will be a 15 minute presentation for each survey, case-study, or research project on December 3rd, 11:00-1:00 (last class) and on December 12th, 12:00-3:00 (final exam time).

Homework Assignments

• Homework 1 Homework 1 due date is extended to Friday, October 17th, 5:00pm.
• Homework 2 Homework 2 due date is extended to Friday, October 31st, 5:00pm.
• Homework 3 Homework 3 due date is Monday, November 24, 5:00pm.
• Homework 4 Homework 4 due date is extended to Friday, December 5th, 5:00pm.

Reading Assignments

• Week 1
  • ``Applying Design by Contract,'' B. Meyer, IEEE Computer, October 1992, pp. 40-51. Available here
  • ``jContractor: Bytecode instrumentation techniques for implementing design by contract in Java.'' In Proceedings of Second Workshop on Runtime Verification, RV 02. Copenhagen, Denmark. July 26, 2002. Available here
• Week 2
  • ``An overview of JML tools and applications.'' Lilian Burdy, Yoonsik Cheon, David R. Cok, Michael D. Ernst, Joseph R. Kiniry, Gary T. Leavens, K. Rustan M. Leino, and Erik Poll. In International Journal on Software Tools for Technology Transfer, 7(3):212-232, June 2005. Available here
  • ``Extended static checking for Java.'' Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, and Raymie Stata. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI 2002). Available here
• Week 3
  • ``Alloy: A Lightweight Object Modeling Notation'' Daniel Jackson. ACM Transactions on Software Engineering and Methodology (TOSEM) Volume 11, Issue 2 (April 2002), pp. 256-290. Available here
• Week 4
  • ``Model Checking Programs.''W. Visser, K. Havelund, G. Brat, S. Park. In Automated Software Engineering Journal Volume 10, Number 2, April 2003. Availablehere
• Week 5
  • ``TestEra: Specification-based Testing of Java Programs Using SAT.'' S. Khurshid and D. Marinov. In Automated Software Engineering Journal, Volume 11, Number 4. October 2004. Availablehere
• Week 6
  • "Korat: Automated Testing Based on Java Predicates." C. Boyapati, S. Khurshid and D. Marinov. ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002), Rome, Italy. Jul 2002. Available here
    Korat Manual
    Korat Tutorial
  • "DART: Directed Automated Random Testing." P. Godefroid, N. Klarlund and K. Sen In Proceedings of PLDI'2005 (ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation), pages 213-223. Availablehere
• Week 7
  • "Application of Design for Verification with Concurrency Controllers to Air Traffic Control Software." Aysu Betin-Can, Tevfik Bultan, Mikael Lindvall, Benjamin Lux, and Stefan Topp. In Proceedings of the 20th IEEE International Conference on Automated Software Engineering (ASE 2005), pp. 14-23, Long Beach, California, USA, November 7-11, 2005. Available here
    
  • "Interface Grammars for Modular Software Model Checking." Graham Hughes and Tevfik Bultan. Proceedings of the 2007 ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2007), pp. 39-49, London, United Kingdom, July 9-12, 2007. Available here
    
• Week 8
  • ``Automatic Extraction of Object-Oriented Component Interfaces,'' J. Whaley, M. C. Martin and M. S. Lam Proceedings of the International Symposium on Software Testing and Analysis, July 2002. Available here
  • ``Synthesis of Interface Specifications for Java Classes,'' R. Alur, Pavol Cerny, P. Madhusudan, and W. Nam. In Proceedings of 32nd Annual ACM SIGPLAN-SIGACT Symp. on Principles of Prog. Languages, POPL'05, Jan , 2005. Available here
• Week 9
  • ``Dynamically discovering likely program invariants to support program evolution'' by Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. IEEE Transactions on Software Engineering, vol. 27, no. 2, Feb. 2001, pp. 1-25. Available here
  • "Simplifying and Isolating Failure-Inducing Input." Andreas Zeller and Ralf Hildebrandt; IEEE Transactions on Software Engineering 28(2), February 2002, pp. 183-200. Available here

List of Topics and Papers

The following list is subject to change.

• Design by Contract
  • ``Applying Design by Contract,'' B. Meyer, IEEE Computer, October 1992, pp. 40-51. Available here
  • Tool: jContractor The tool and the following papers are available here . Installation help for jContractor can be referenced here
    • ``jContractor: A Reflective Java Library to Support Design By Contract.'' In Proceedings of Meta-Level Architectures and Reflection, 2nd International Conference, Reflection '99. Saint-Malo, France, July 1999. Lecture Notes in Computer Science #1616, Springer Verlag, July 1999, pp.175-196. JContractor paper and tool
    • ``jContractor: Bytecode instrumentation techniques for implementing design by contract in Java.'' In Proceedings of Second Workshop on Runtime Verification, RV 02. Copenhagen, Denmark. July 26, 2002.
• Java Modeling Language (JML) and Extended Static Checking
  • Tool: ESC/Java The tool is available here and the following paper is available here
    • ``Extended static checking for Java.'' Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, and Raymie Stata. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI 2002).
  • The Java Modeling Language (JML) Tools The website for the JML project and asociated tools and papers are here
    • ``An overview of JML tools and applications.'' Lilian Burdy, Yoonsik Cheon, David R. Cok, Michael D. Ernst, Joseph R. Kiniry, Gary T. Leavens, K. Rustan M. Leino, and Erik Poll. In International Journal on Software Tools for Technology Transfer, 7(3):212-232, June 2005.Download the paper here
• Object Oriented Modeling
  • Unified Modeling Language (UML) and Object Constraint Language (OCL)
    • Object Management Group's UML resource page is here
    • Object Constraint Language specification is available here
  • Tool: USE The tool and the following paper are available here
    • ``Validating OCL Specifications with the USE Tool - An Example Based on the BART Case Study.'' Paul Ziemann and Martin Gogolla. In Proc. 8th Int. Workshop Formal Methods for Industrial Critical Systems (FMICS'2003). Electronic Notes in Theoretical Computer Science (ENTCS), Vol. 80, Elsevier, 2003. Download here
  • Tool: Alloy Analyzer The tool is available here and the following papers are available here
    • ``Alloy: A Lightweight Object Modeling Notation'' Daniel Jackson. ACM Transactions on Software Engineering and Methodology (TOSEM) Volume 11, Issue 2 (April 2002), pp. 256-290. Download the paper here
    • ``A Micromodularity Mechanism,'' Daniel Jackson, Ilya Shlyakhter and Manu Sridharan. Proc. ACM SIGSOFT Conf. Foundations of Software Engineering/European Software Engineering Conference (FSE/ESEC '01), Vienna, September 2001. Download the paper here
• Software Model Checking
  • Tool: Java Pathfinder Available here
    • ``Model Checking Programs.'' W. Visser, K. Havelund, G. Brat, S. Park. In Automated Software Engineering Journal Volume 10, Number 2, April 2003. Available here
  • ``eXplode: a Lightweight, General System for Finding Serious Storage System Errors,'' Junfeng Yang, Can Sar, and Dawson Engler. Proceedings of the 7th Symposium on Operating System Design and Implementation, 2006. Available here
  • "Saturn: A Scalable Framework for Error Detection using Boolean Satisfiability." Y. Xie and A. Aiken. To appear in ACM Transactions on Programming Languages and Systems. Available here
• Automated Testing
  • "Software Assurance by Bounded Exhaustive Testing," D. Coppit, J. Yang, S. Khurshid, W. Le, and K. Sullivan. IEEE Transactions on Software Engineering (TSE), Volume 31, Issue 4, Pages 328-339, April 2005. Available here
  • "TestEra: A Novel Framework for Automated Testing of Java Programs." D. Marinov and S. Khurshid. 16th IEEE Conference on Automated Software Engineeri ng (ASE 2001), San Diego, CA. Nov 2001. Available here
  • "Korat: Automated Testing Based on Java Predicates." C. Boyapati, S. Khurshid and D. Marinov. ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002), Rome, Italy. Jul 2002. Available here
  • "DART: Directed Automated Random Testing," Patrice Godefroid, Nils Klarlund, and Koushik Sen, In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI 2005). Available here
  • Tool: CUTE Available here
    • "CUTE: A Concolic Unit Testing Engine for C," Koushik Sen, Darko Marinov, and Gul Agha. In Proceedings of the 2005 European Software Engineering Conference (ESEC) and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE) (ESEC/FSE'05). Available here
  • "Designing and comparing automated test oracles for GUI-based software applications" by Qing Xie and Atif M. Memon. ACM Transactions on Software Engineering and Methodology, vol. 16, no. 1, 2007, ACM Press. Available here
  • "Model-Based Testing of Community-Driven Open-Source GUI Applications" In ICSM '06: Proceedings of the 22nd IEEE International Conference on Software Maintenance, (Washington, DC, USA), 2006, pp. 145-154. Available here
• Software Interfaces
  • ``Automatic Extraction of Object-Oriented Component Interfaces,'' J. Whaley, M. C. Martin and M. S. Lam Proceedings of the International Symposium on Software Testing and Analysis, July 2002. Available here
  • ``Synthesis of Interface Specifications for Java Classes,'' R. Alur, Pavol Cerny, P. Madhusudan, and W. Nam. In Proceedings of 32nd Annual ACM SIGPLAN-SIGACT Symp. on Principles of Prog. Languages, POPL'05, Jan , 2005. Available here
  • "Automated Environment Generation for Software Model Checking," Oksana Tkachuk, Matthew B. Dwyer, Corina Pasareanu. In the Proceedings of the 18th IEEE International Conference on Automated Software Engineering (ASE 2003), May 2003. Available here
  • ``Interface Grammars for Modular Software Model Checking,'' G. Hughes and T. Bultan. To appear in the Proceedings of ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2007). Available here
• Dynamic Analysis
  • Tool: Daikon Tool is available here and the publications below are available here
    • ``Dynamically discovering likely program invariants to support program evolution'' by Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. IEEE Transactions on Software Engineering, vol. 27, no. 2, Feb. 2001, pp. 1-25.Available here
    • "The Daikon system for dynamic detection of likely invariants." by Michael D. Ernst, Jeff H. Perkins, Philip J. Guo, Stephen McCamant, Carlos Pacheco, Matthew S. Tschantz, and Chen Xiao. Science of Computer Programming, 2007.Available here
  • Runtime monitoring
    • "Checking and correcting behaviors of Java programs at runtime with JavaMOP." F. Chen, M. D'Amorim, and G. Rosu. In RV'05, volume 144(4) of ENTCS, 2005. Available here
    • "Synthesizing Monitors for Safety Properties." K. Havelund and G. Rosu. Proceedings of the 8th International Conference on Tools and Algorithms for Construction and Analysis of Systems, Grenoble, France, Vol. 2280, pp. 342-356, April 2002. Availale here
• Automated Debugging
  • "Simplifying and Isolating Failure-Inducing Input." Andreas Zeller and Ralf Hildebrandt; IEEE Transactions on Software Engineering 28(2), February 2002, pp. 183-200. Available here
  • "Yesterday, my program worked. Today, it does not. Why?" Andreas Zeller; Proc. ESEC/FSE 99, Toulouse, France, September 1999, Vol. 1687 of LNCS, pp. 253-267. Available here
  • "Locating Causes of Program Failures," by Holger Cleve, Andreas Zeller. ICSE '05: Proceedings of the 27th international conference on Software engineering, Pages 342-351, May 2005. Available here
  • "Bug Isolation via Remote Program Sampling," Ben Liblit, Alex Aiken, Alice X. Zheng, and Michael I. Jordan. ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation (PLDI 2003). here
  • "Scalable Statistical Bug Isolation," Ben Liblit, Mayur Naik, Alice X. Zheng, Alex Aiken, and Michael I. Jordan. ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI 2005). here

Lecture Notes

• Lecture 1: Introduction
• Lecture 2: Design by Contract
• Lecture 3: Java Modeling Language and Extended Static Checking
• Lecture 4: Object Constraint Language
• Lectures 5, 6 and 7: Alloy
• Lectures 8 and 9: Model Checking Programs
• Lectures 10 and 11: Testing, Automated Testing
• Lecture 12: Directed Automated Random Testing and Concolic Execution. See Koushik Sen's slides for his PLDI'05 and ESEC/FSE'05 papers here.
• Lectures 13 and 14: Interfaces, Modularity, and Verification
• Lecture 15: Interface Extraction
• Lecture 16: Automatically Discovering Likely Invariants
• Lecture 17: Automated Debugging