Basic Commands of KAFKA and ZOOKEEPER

How to check the brokers state?

echo stat | nc localhost 2181

echo dump | nc localhost 2181 | grep brokers

How to create topic?

bin/kafka-topics.sh   –bootstrap-server  localhost:9092   –create –topic   topicname  –partitions  1  –replication-factor 1

How to list all the topics?

bin/kafka-topics.sh   –bootstrap-server  localhost:9092  –list

How to describe topic?

bin/kafka-topics.sh   –bootstrap-server  localhost:9092  –describe   –topic  topicname

How to run producer?

bin/kafka-console-producer.sh –bootstrap-server localhost:9092 –topic mytopic

How to run consumer?

bin/kafka-console-consumer.sh –bootstrap-server localhost:9092 –topic mytopic –from-beginning

How to check list of consumer-groups?

bin/kafka-consumer-groups.sh –bootstrap-server localhost:9092 –-list

How to describe consumer-groups?

bin/kafka-consumer-groups.sh –bootstrap-server localhost:9092 –describe -–group groupname

How to run consumer with own consumer group?

bin/kafka-console-consumer.sh –bootstrap-server localhost:9092 –topic mytopic –group my-group

How to increase partition of topics?

bin/kafka-topic.sh  –bootstrap-server localhost:9092 –alter –topic topicname  –partitions 2

How to move or reassgin partition?

We have to run three commands for reassignments and create json file with this content.

{“topcis”: [{“topic”: “test”}],

“version”:1

}

bin/kafka-reassign-partitions.sh –zookeeper localhost:2181 –topics-to-move-json-file topicsToMove.json  –broker-list “2,3” –generate

bin/kafka-reassign-partitions.sh –zookeeper localhost:2181 –reassignment-json-file  suggestedChange.json –execute

bin/kafka-reassign-partitions.sh –zookeeper localhost:2181 –reassignment-json-file  suggestedChange.json –verify

How we can do changes in producer.config?

We can do these kind of changes which Is very important for the throughput.

  • Compression.type
  • Batch.size
  • Linger.ms
  • Buffer.memory
  • Max.request.size

Important points for consumer.config file for good throughput

  • Heartbeat.interval.ms
  • Session.timeout.ms
  • Max.partition.fetch.bytes
  • Fetch.max.bytes

How to create producer-console with particular configuration?

bin/kafka-console-producer.sh –bootstrap-server localhost:9092 –topic mytopic –producer.config config/producer.config

 

How to install APACHE KAFKA on CentOS 7

Leave a Reply

Your email address will not be published. Required fields are marked *