CSC 536 Distributed Systems II
Summary
Distributed Systems II (DSII) is an advanced distributed systems course. While Distributed Systems I (CSC 435) covers the plumbing of distributed systems (e.g., processes, networking, and naming), DSII covers the techniques required to build large-scale, highly available, fault tolerant distributed systems.
In DSII, we will cover classical distributed algorithms that solve fundamental problems in distributed systems including synchronization, distributed transactions, replication and replica consistency, fault tolerance, reliable group communication, and crash recovery. Examples of algorithms that we will cover include vector clocks, used by Amazon to ensure high availability, consistent hashing, used by Akamai to assign replicated web content to web servers, the gossip protocol, used by Facebook's Cassandra to propagate updates to replicas, Merkle trees, used by Bitcoin to ensure efficient and secure verification of bitcoin transactions, and the Paxos/Raft algorithms, used by Google to provide fault tolerance across its data storage services.
We will focus in particular on problems and solutions relevant to systems that store and process massive data sets (Big Data). We will go over the design decisions behind distributed database systems such as Facebook's Cassandra, Amazon's Dynamo, and Google's BigTable and recently developed Spanner. In the process, we will see that in each case the design decisions were driven by relatively narrow goals (e.g., highly available shopping carts for Amazon and web search data batch processing for Google) and that this is in general a wise approach for system development.
To get our hands dirty and implement the distributed algorithms covered in class, we will use Akka, a Scala/Java toolkit for developing highly concurrent, distributed, scalable, fault tolerant event-driven applications. We will also understand the role of bleeding edge distributed system technologies such as docker and etcd for distributed application development and deployment. For your final project, you will have the opportunity to build a distributed system prototype using Akka and such tools.
Texts
Required
A.S. Tanenbaum and M. van Steen, Distributed Systems: Principles and Paradigms, 2nd edition, Prentice-Hall, 2006.
Optional:
Vaughn Vernon: Reactive Messaging Patterns with the Actor Model: Applications and Integration in Scala and Akka, Addison-Wesley Professional, 2015.
R. Kuhn, B. Hanafee, and J. Allen, Reactive Design Patterns, Manning Publications, 2017 .
Grading
The course grade will be apportioned as follows:
| Homeworks |
60% |
| Project |
40% |
There will be a homework due most weeks. A homework or project not handed in by the deadline will receive 0 points: this is because I want to discuss the homework solutions with the class as soon as possible. Because it is unrealistic to expect that you will be able to submit every homework on time, I will drop the lowest 2 homework grades when computing your overall homework grade.
Prerequisites
You must have taken SE435, or equivalent courses in distributed systems. I will assume that:
- you know how to create, debug, compile and run Java and Scala code on the platform of your choice;
- you use a reasonable programming style (i.e. your code is easy to read and concise) ;
- you know and understand the basic architectures of distributed systems, and the basics of networking, operating systems (processes, threads), naming, and security.
This syllabus is subject to change as necessary during the quarter. If a change occurs, it will be thoroughly addressed during class, posted under Announcements in D2L and sent via email.
Evaluations are a way for students to provide valuable feedback regarding their instructor and the course. Detailed feedback will enable the instructor to continuously tailor teaching methods and course
content to meet the learning goals of the course and the academic needs of the students. They are a requirement of the course and are key to continue to provide you with the highest quality of teaching. The
evaluations are anonymous; the instructor and administration do not track who entered what responses. A program is used to check if the student completed the evaluations, but the evaluation is completely
separate from the student’s identity. Since 100% participation is our goal, students are sent periodic reminders over three weeks. Students do not receive reminders once they complete the evaluation.
Students complete the evaluation online in CampusConnect.
This course will be subject to the university's academic integrity policy. More information can be found at http://academicintegrity.depaul.edu/ If you
have any questions be sure to consult with your professor.
All students are expected to abide by the University's Academic Integrity Policy which prohibits cheating and other misconduct in student coursework. Publicly sharing or posting online any prior or current materials from this course (including exam questions or answers), is considered to be providing unauthorized assistance prohibited by the policy. Both students who share/post and students who access or use such materials are considered to be cheating under the Policy and will be subject to sanctions for violations of Academic Integrity.
All students are required to manage their class schedules each term in accordance with the deadlines for enrolling and withdrawing as indicated in the University Academic Calendar. Information on enrollment, withdrawal, grading and incompletes can be found at http://www.cdm.depaul.edu/Current%20Students/Pages/PoliciesandProcedures.aspx.
Students who feel they may need an accommodation based on the impact of a disability should contact the instructor privately to discuss their specific needs. All discussions will remain confidential.
To ensure that you receive the most appropriate accommodation based on your needs, contact the instructor as early as possible in the quarter (preferably within the first week of class), and make sure that
you have contacted the Center for Students with Disabilities (CSD) at:
Lewis Center 1420, 25 East Jackson Blvd.
Phone number: (312)362-8002
Fax: (312)362-6544
TTY: (773)325.7296