DDIA - Chapter 9 - Consistency and Consensus - Thoughts and notes

Search for a command to run...

No comments yet. Be the first to comment.
One of the methods of abstraction is to integrate another service. But can you trust someone else's service? Third-party APIs are notorious for failing at the worst possible time. Over time, I ended u

A race condition can hit you anytime, you think it won't because your system isn't that big, or doesn't process a lot of requests, but it will happen, and when it happens, you will feel a bit silly ab

The world is going through a tumultuous phase, but that doesn't stop the progress of humanity. Things go on, people move, prices fluctuate, we eat fruit and live a healthy life. April is hot, so hot,

Well, I celebrated my birthday. :P I worked on a functionality end to end using AI. It was a simple feature and AI wrote everything correctly in a single pass. I used Codex. After that, I had to ask i

Let me think. So I worked on a new functionality and raised a PR in just 2 days for the whole end-to-end correct flow. I did not focus much on performance because I wanted to first get it right and th

Disclaimer: this blog has been written with the help of LLM, please do not hate me
I never imagined that consensus would be such a difficult problem, in a democracy one has consensus right, it is so easy. Even if there are multiple failure points the leader gets elected.
Anyway, this chapter speaks a lot about linearisability, consistency, consensus, how to be fault tolerant during consensus. Thing is that the tradeoff with making the system consistent is performance. There wasn’t much to discover in this chapter, I did find it a bit boring. The basic idea while building a distributed system should be that time is linear. I’d always use epochs.
Makes a distributed system appear as a single replica without replication lag
Ensures all clients see the most recent value once it's written
Requires consistent data across all replicas at the same time
Strict consistency model
Once a client sees a new value, all other clients must see that same new value
Critical for scenarios like:
Leader election in single-leader systems
Choosing unique usernames
Seat booking systems
Performance-intensive
Often sacrificed for availability
Costly in multi-data center environments
Not many systems provide true linearizability
High performance overhead
Difficult to implement across distributed systems
Network delays can significantly impact implementation
Database-internal transactions
Used within same database system
Easier to implement
Heterogeneous distributed transactions
Involve multiple, different systems
Focus on ensuring atomic commits
Two phases: Prepare and Commit
Transaction coordinator manages process
Ensures all nodes commit or abort together
Performance bottlenecks
Complexity of coordinating across systems
Potential for system-wide blocking if coordinator fails
Getting multiple nodes to agree on a value with specific properties:
Uniform agreement
Integrity
Validity
Termination
Requires majority of nodes to be functioning
Uses epoch numbering for leader selection
Implements total order broadcast
Requires strict majority of nodes
Difficult to add/remove nodes dynamically
Relies on timeout mechanisms
Performance can be impacted by frequent leader elections
Linearizable atomic operations
Total ordering of operations
Failure detection
Change notifications
Service discovery