Step By Step : Installing Kafka in Mac

  1. Open “Terminal” app from Applications or Command + Space and then type “Terminal” press Enter/Return Key
  2. Install Homebrew (Copy / Paste the following command in the Terminal window and press enter)
    1. ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null
  3. Now install Kafka(Copy / Paste the following command in the Terminal window and press enter)
    • brew install kafka
    • Sample Output of above command :
    • ==> Installing dependencies for kafka: zookeeper
      
      ==> Installing kafka dependency: zookeeper
      
      ==> Downloading https://homebrew.bintray.com/bottles/zookeeper-3.4.10.sierra.bottle.tar.gz
      
      ######################################################################## 100.0%
      
      ==> Pouring zookeeper-3.4.10.sierra.bottle.tar.gz
      
      ==> Caveats
      
      To have launchd start zookeeper now and restart at login:
      
       brew services start zookeeper
      
      Or, if you don't want/need a background service you can just run:
      
       zkServer start
      
      ==> Summary
      
      🍺 /usr/local/Cellar/zookeeper/3.4.10: 241 files, 31.4MB
      
      ==> Installing kafka
      
      ==> Downloading https://homebrew.bintray.com/bottles/kafka-0.11.0.0.sierra.bottle.tar.gz
      
      ######################################################################## 100.0%
  4. Now list all services
    • brew services list
    • Sample output of above command
    • spradhan-macos:/ sudhir.pradhan$ brew services list
      Name Status User Plist
      kafka stopped
      mysql stopped
      postgresql started sudhir.pradhan /Users/sudhir.pradhan/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
      presto stopped
      redis started sudhir.pradhan /Users/sudhir.pradhan/Library/LaunchAgents/homebrew.mxcl.redis.plist
      zookeeper stopped
    • From above output it seems the kafka installed but not running
    • So before starting Kafka, make sure we the ZooKeeper is running
  5. Lets start ZooKeeper
    • brew services start zookeeper
    • Sample output
      1. spradhan-macos:/ sudhir.pradhan$ brew services start zookeeper
        ==> Successfully started `zookeeper` (label: homebrew.mxcl.zookeeper)
  6. Now start Kafka (From above output we made sure the Zookeeper is running)
    • brew services start kafka
    • Sample output
      1. spradhan-macos:/ sudhir.pradhan$ brew services start kafka
        ==> Successfully started `kafka` (label: homebrew.mxcl.kafka)
  7. Lets verify if our required services are running
    • brew services list
    • Sample output
    • spradhan-macos:/ sudhir.pradhan$ brew services list
      Name Status User Plist
      kafka started sudhir.pradhan /Users/sudhir.pradhan/Library/LaunchAgents/homebrew.mxcl.kafka.plist
      mysql stopped
      postgresql started sudhir.pradhan /Users/sudhir.pradhan/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
      presto stopped
      redis started sudhir.pradhan /Users/sudhir.pradhan/Library/LaunchAgents/homebrew.mxcl.redis.plist
      zookeeper started sudhir.pradhan /Users/sudhir.pradhan/Library/LaunchAgents/homebrew.mxcl.zookeeper.plist
  8. Lets create a topic named “test”
    • kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 4 --topic test
    • Sample output
    • spradhan-macos:/ sudhir.pradhan$ kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 4 --topic test
      Created topic "test".
  9. Desctibe above created topic
    • kafka-topics.sh --describe --zookeeper localhost:2181 --topic
    • Sample output
    • spradhan-macos:/ sudhir.pradhan$ kafka-topics –describe –zookeeper localhost:2181 –topic test

    • Topic:test PartitionCount:4 ReplicationFactor:1 Configs:

    • Topic: test Partition: 0 Leader: 0 Replicas: 0 Isr: 0

    • Topic: test Partition: 1 Leader: 0 Replicas: 0 Isr: 0

    • Topic: test Partition: 2 Leader: 0 Replicas: 0 Isr: 0

    • Topic: test Partition: 3 Leader: 0 Replicas: 0 Isr: 0

  10. Start Producer and Consumer in 2 different terminal window
    • in terminal 1:
      1. kafka-console-producer --broker-list localhost:9092 --topic test
    • in terminal 2:
      1. kafka-console-consumer --broker-list localhost:9092 --topic test
  11. Now Publish message to the above topic in Terminal 1
    1. spradhan-macos:/ sudhir.pradhan$ kafka-console-producer –broker-list localhost:9092 –topic test
      >Hi There
      >

  12. Now check in Terminal 1 if the consumer received the message
    1. spradhan-macos:/ sudhir.pradhan$ kafka-console-consumer –bootstrap-server localhost:9092 –from-beginning –topic test
      Hi There

    2. Yehhhh …  there you go

 

Hope you enjoyed the article. Please like or comment.

Advertisements