Everything you ever wanted to know about Zookeeper

This blog post will discuss the research paper, Zookeeper: Wait-free coordination for Internet-scale systems. Click here for downloading the paper.

What problem does Zookeeper solve?

Coordination is of utmost importance in distributed applications. While building distributed systems, we need primitives like distributed lock services, shared registers, group messaging, etc. Instead of implementing these on our own, Zookeeper provides such primitives off-the-shelf.

Keywords

Shared Registers

Distributed lock services

Distributed locks are required so that the system can agree on a particular state at any given time. They are required to maintain data integrity.

Wait-free mechanism

Clients’ requests are returned to them from the server and they(clients) don’t have to wait for the other slow clients and servers.

Event-driven mechanism

When the flow of control or flow of data is driven by the changes in state, that architecture is event-driven architecture.

Linearizability

Source: Wikipedia

Zookeeper Design

Just a bunch of computers(commodity hardware) using Zookeeper API to have one leader.
Figure 1 from the paper

Zookeeper as a Service

Client — user of ZK service

Figure 1 from the paper

Client APIs

  • create(path, data, flags); • delete(path, version); • exists(path, watch); • getData(path, watch); • setData(path, data, version); • getChildren(path, watch); • sync(path);

Zookeeper Guarantees

Linearizable writes

Explained above in detail, the leader ensures that all the writes are processed in the order. Servers send write requests to leader and leader enforces them in their sequence of arrival.

FIFO Client Order

The client’s requests are processed in the exact same order as generated.

Atomicity

Just like other consensus protocols(Raft, Paxos, etc), all the updates in the system are transactional, there is no partial update.

Reliability

All updates that are successful will persist as they are applied to the majority and they will never be rolled back.

Scenarios to understand Leader election and configuration management in Zookeeper

Consider there are one leader and 3 servers(followers) in the current ZK system. Suddenly, the leader dies due to some reason. Now, the new leader will be elected but it has to modify all the configuration files before any worker reads them. To ensure this, ZK makes the workers read the config only if there exists a “ready” znode. So, when the leader dies, the new one gets elected and it deletes the ready znode at first. This makes the configuration files inaccessible by the workers. Now, the new leader updates the configuration and then creates the ready znode following which all workers are able to read the updated configuration.

Zookeeper as a Lock Service

This is how ZK implements a single lock:

Summary

Figure 4 from the paper

Code + Data.