Here’s a blog entry on how to get started using Apache Kafka, a high-throughput, distributed, publish-subscribe messaging system like JMS or ActiveMQ. Very simple to use interface. Sample code is easy to run.
Only caveat is that your
/etc/hosts file (or comparable) will need to have an entry for your host name. If you’ve named your machine
foo, then you’ll need an entry in your hosts file that reads:
Here’s the article:
Here’s my summary of the article:
Getting Started with Kafka
To start the server, first you have to start the included configuration manager,
ZooKeeper, and then run Kafka itself using the ZooKeeper managed configuration.
(If you get errors about an unresolved host name, ensure that the /etc/hosts
file contains a record that specifies the current machine’s host name. For
Setup Step 0:
cd [kafka installation directory]
Setup Step 1:
bin/zookeeper-server-start.sh config/zookeeper.properties &
Setup Step 2:
Setup Step 3 -- create some topics:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic fast-messages
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic summary-markers
Setup Step 4 -- list the topics, because you can:
bin/kafka-topics.sh --list --zookeeper localhost:2181
Now you can run your app.
Demo Step 0:
You can run some example programs. This example has a simple producer and consumer.
git clone https://github.com/mapr-demos/kafka-sample-programs.git
mvn clean package
Demo Step 1:
Start the consumer:
Demo Step 2:
Then run the producer as often as you want to see all the fun:
Writing a consumer or producer requires the following Maven dependency:
The consumer has a dependency on a histogram library to produce some interesting output.
That's about all you need to run and try out Kafka.