Big Data > Kafka for Streaming Data > What is Zookeeper?

What is Zookeeper?

Apache ZooKeeper was first developed by Yahoo! in the mid-2000s. Yahoo created it to help manage distributed applications in a reliable and scalable way. The main goal was to build a central coordination service that could keep different parts of a system in sync, detect failures, and manage settings easily. Later, Yahoo gave ZooKeeper to the Apache Software Foundation, and it became a top-level Apache project in 2010. Since then, ZooKeeper has been used widely in the big data world. It has become an important part of many systems like Apache Kafka, Apache HBase, Apache Hadoop, and Apache Solr.

In Apache Kafka, ZooKeeper has a key role. It works as a central service that stores configuration data, helps manage the system, and supports tasks like leader election and synchronization between different parts of the Kafka cluster.

 

What are the functions of ZooKeeeper in Apache Kapka?

ZooKeeper plays a crucial role in managing and coordinating the Kafka cluster. The following are the main functions of ZooKeeper in Apache Kafka:

• ZooKeeper maintains information about Kafka brokers (nodes) in the cluster. It helps keep track of controller broker and brokers that are currently alive
• ZooKeeper helps in electing a controller broker in case of failure of any leader broker.
• ZooKeeper stores metadata such as Topics and partitions, Broker IDs, Access Control Lists (ACLs)
• All the configuration of Kafka nodes has written in the configuration settings of ZooKeeper, ensuring consistent configurations across the cluster.
• ZooKeeper provides a watcher mechanism when a change occurs in the total system, it notifies the interested components like brokers or the controller.

 

Prev

Used Terms in Kafka

Next

An Illustration of Kafka

Feedback

Submit