A tool for managing Apache Kafka.
It supports the following :
Consumer List View
Consumed Topic View
The minimum configuration is the zookeeper hosts which are to be used for kafka manager state. This can be found in the application.conf file in conf directory. The same file will be packaged in the distribution zip file; you may modify settings after unzipping the file on the desired server.
You can specify multiple zookeeper hosts by comma delimiting them, like so:
Alternatively, use the environment variable
ZK_HOSTS if you don't want to hardcode any values.
You can optionally enable/disable the following functionality by modifying the default list in application.conf :
Consider setting these parameters for larger clusters with jmx enabled :
Here is an example for a kafka cluster with 10 brokers, 100 topics, with each topic having 10 partitions giving 1000 total partitions with JMX enabled :
The follow control consumer offset cache's thread pool and queue :
You should increase the above for large # of consumers with consumer polling enabled. Though it mainly affects ZK based consumer polling.
Kafka managed consumer offset is now consumed by KafkaManagedOffsetCache from the "__consumer_offsets" topic. Note, this has not been tested with large number of offsets being tracked. There is a single thread per cluster consuming this topic so it may not be able to keep up on large # of offsets being pushed to the topic.
The command below will create a zip file which can be used to deploy the application.
./sbt clean dist
Please refer to play framework documentation on production deployment/configuration.
If java is not in your path, or you need to build against a specific java version, please use the following (the example assumes oracle java8):
$ PATH=/usr/local/oracle-java-8/bin:$PATH \ JAVA_HOME=/usr/local/oracle-java-8 \ /path/to/sbt -java-home /usr/local/oracle-java-8 clean dist
This ensures that the 'java' and 'javac' binaries in your path are first looked up in the oracle java8 release. Next, for all downstream tools that only listen to JAVA_HOME, it points them to the oracle java8 location. Lastly, it tells sbt to use the oracle java8 location as well.
After extracting the produced zipfile, and changing the working directory to it, you can run the service like this:
By default, it will choose port 9000. This is overridable, as is the location of the configuration file. For example:
$ bin/kafka-manager -Dconfig.file=/path/to/application.conf -Dhttp.port=8080
Again, if java is not in your path, or you need to run against a different version of java, add the -java-home option as follows:
$ bin/kafka-manager -java-home /usr/local/oracle-java-8
If you'd like to create a Debian or RPM package instead, you can run one of:
sbt debian:packageBin sbt rpm:packageBin
Logo/favicon used is from Apache Kafka.
Apache Licensed. See accompanying LICENSE file.