Apache Storm Integration With Apache Kafka


Installing Apache Storm

The prerequisite for storm to work on the machine.
a. Download and installation commands for ZeroMQ 2.1.7:
Run the following commands on terminals

b. Download and installation commands for JZMQ: 

 

2. Download latest storm from http://storm.apache.org/downloads.html 

Second start Storm Cluster by starting master and worker nodes.

Start master node i.e. nimbus.

To start master i.e. nimbus go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]

Start worker node i.e. supervisor.

To start worker i.e. supervisor go to the ‘bin’ directory of the Storm installation and execute following command. [separate command line window]

Apache Kafka + Apache Storm

1. Kafka Producer

2. Storm Spout/Topology

3. Bolts

4. SentenceBolt

Running the Kafka and storm application

1. First create Kafka topic “words_topic”
Start server:

Create topic:

Enter something on Producer console

kafka+storm

Run the strom jar.

kafka+storm1

 

 

 

 

 

 

 

 

 

 

 


Profile photo of Siva

About Siva

Senior Hadoop developer with 4 years of experience in designing and architecture solutions for the Big Data domain and has been involved with several complex engagements. Technical strengths include Hadoop, YARN, Mapreduce, Hive, Sqoop, Flume, Pig, HBase, Phoenix, Oozie, Falcon, Kafka, Storm, Spark, MySQL and Java.

Leave a comment

Your email address will not be published. Required fields are marked *