Big Data > Kafka for Streaming Data > An Illustration of Kafka

1. Kapka Installation

Download Kafka from: https://kafka.apache.org/quickstart and extract it in your c drive.

 

2. Checking to the folder

Go to the folder

 

The bin folder contains some command line of Kapka tools:

The config folder contains all the configuration settings

The libs folder contains all the “.ger” files

3. Starting Kafka

Since Kapka uses Zookeeper we have to activate Zookeeper first. It is available in the bin folder. To start Zookeeper open a command terminal:

         .\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

The above command has one parameter only “.\config\zookeeper.properties” this is default configuration.

Here we will see the port number where bind it information, normally it is 2181

4. Starting Kafka Broker.

To start Kafka Broker with default configuration open a command terminal:

          .\bin\windows\kafka-server-start.bat .\config\server.properties

A lot of information will appear and an Apache server has been started. Make sure that there are no errors.

5. Creating Kafka topic

Kafka creates topic automatically but we want to give a name here.  When a producer creates a message when there is no existence of any topic, kafka creates a topic and accept the message. Let us create a topic using topic manage tool:

          .\bin\windows\kafka-topics.bat --bootstrap-server localhost:2181

                    --create --topic patwary --partitions 2 --replication-factor 1

1. First parameter is zookeeper server address and port number
2. Second parameter is After create command with topic name, partition number, and replication factors.

Here we have a single broker but how is it possible to create 2 partitions? This is not a problem at all. . Kafka will distribute partitions among all of its brokers. So both the partions will be created into one broker.

6. Open Two Terminals

Now we need to open two cmd terminals one for producer and another for consumer using default port 9091 or 9092.

.\bin\windows\kafka-console-producer.bat --bootstrap-server localhost:9092 --topic patwary

In another terminal

.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic patwary

 

 

 

Prev

What is Zookeeper?

No More

Feedback

Submit