Distributed systems help programmers aggregate the resources of many networked computers to construct highly available and scalable services. Most of the applications and services we interact with today are distributed, some at enormous scales. Building large-scale, reliable distributed systems is hard, and has been the topic of enormous amounts of research and development.
This class provides an overview of influential research that provided the basis of most large-scale, cloud infrastructures today. Students read and discuss papers on important distributed systems topics, including distributed consensus, consistency models and algorithms, service-oriented architectures, large-scale data storage, and distributed transactions, and big-data processing frameworks.
In addition, students form teams to work on a semester-long research project that will give them hands-on experience with building or extending a distributed system. At the class's end, students should have a good understanding of key research topics in distributed systems, as well as roughly how to go about developing such topics on their own.
Papers: you will be responsible for reading two-three papers as assigned before each class, and contributing your thoughts on each assigned paper to the class discussion board before the class that covers it. Paper in-class discussion will account for 20% of the grade.
Quizzes: there are going to be surprise in-class quizzes that will account for 10% of the grade.
Research project: you will work in teams of 2-3 (depends on class size) students to develop a research project throughout the course of the semester. Any interesting research project in the broad distributed systems area will be considered, but will have to be approved by the staff before you can undertake it. The project, along with its presentations and document accounts for 70% of the grade.
The project will require a lot of coding, as you will have to build or extend a distributed system. Hence, in this class, we require that you have solid coding experience, particularly building systems-level components (e.g., not just apps). This can translate either to personal experience or industry experience or to the following Columbia courses or equivalents:
This class is very closely inspired by the Distributed Systems graduate class at the University of Washington, particularly the 2013 edition by Steve Gribble.