How to check the brokers state?
echo stat | nc localhost 2181
echo dump | nc localhost 2181 | grep brokers
How to create topic?
bin/kafka-topics.sh –bootstrap-server localhost:9092 –create –topic topicname –partitions 1 –replication-factor 1
How to list all the topics?
bin/kafka-topics.sh –bootstrap-server localhost:9092 –list
How to describe topic?
bin/kafka-topics.sh –bootstrap-server localhost:9092 –describe –topic topicname
How to run producer?
bin/kafka-console-producer.sh –bootstrap-server localhost:9092 –topic mytopic
How to run consumer?
bin/kafka-console-consumer.sh –bootstrap-server localhost:9092 –topic mytopic –from-beginning
How to check list of consumer-groups?
bin/kafka-consumer-groups.sh –bootstrap-server localhost:9092 –-list
How to describe consumer-groups?
bin/kafka-consumer-groups.sh –bootstrap-server localhost:9092 –describe -–group groupname
How to run consumer with own consumer group?
bin/kafka-console-consumer.sh –bootstrap-server localhost:9092 –topic mytopic –group my-group
How to increase partition of topics?
bin/kafka-topic.sh –bootstrap-server localhost:9092 –alter –topic topicname –partitions 2
How to move or reassgin partition?
We have to run three commands for reassignments and create json file with this content.
{“topcis”: [{“topic”: “test”}],
“version”:1
}
bin/kafka-reassign-partitions.sh –zookeeper localhost:2181 –topics-to-move-json-file topicsToMove.json –broker-list “2,3” –generate
bin/kafka-reassign-partitions.sh –zookeeper localhost:2181 –reassignment-json-file suggestedChange.json –execute
bin/kafka-reassign-partitions.sh –zookeeper localhost:2181 –reassignment-json-file suggestedChange.json –verify
How we can do changes in producer.config?
We can do these kind of changes which Is very important for the throughput.
- Compression.type
- Batch.size
- Linger.ms
- Buffer.memory
- Max.request.size
Important points for consumer.config file for good throughput
- Heartbeat.interval.ms
- Session.timeout.ms
- Max.partition.fetch.bytes
- Fetch.max.bytes
How to create producer-console with particular configuration?
bin/kafka-console-producer.sh –bootstrap-server localhost:9092 –topic mytopic –producer.config config/producer.config