In this post, we briefly discuss about Apache Flume Installation and Configuration on Ubuntu machine.

The current version of Apache Flume is called as Flume NG (Next Generation) and it’s old version is renamed as Flume OG (Old Generation). In this post, we will discuss about Flume NG only.

Prerequisite: 

1. JDK 1.6 or later versions of Java installed on our Ubuntu machine.
2. Memory – Sufficient memory for configurations used by sources, channels or sinks.
3. Disk Space – Sufficient disk space for configurations used by channels or sinks.
4. Directory Permissions – Read/Write permissions for directories used by agent.

Apache Flume Installation On Ubuntu:

• Download latest stable release of apache flume binary distribution from apache download mirrors at http://flume.apache.org/download.html . At the time of writing this post, apache-flume-1.5.0 is the latest version and the same (apache-flume-1.5.0.1-bin.tar.gz) is used for installation in this post.

• Copy the apache-flume-1.5.0.1-bin.tar.gz from downloads folder to our preferred flume installation directory, usually into /usr/lib/flume and unpack the tarball. Below are the set of commands to perform these activities.

 And below is the screen shot from the installation terminal.

• Set FLUME_HOME, FLUME_CONF_DIR environment variables in .bashrc file as shown below and add the Flume bin directory to PATH environment variable.

• In FLUME_CONF_DIR directory, rename flume-env.sh.template file to flume-env.sh and provide value for JAVA_HOME environment variable with Java installation directory.
• If we are going to use memory channels while setting flume agents, it is preferable to increase the memory limits in JAVA_OPTS variable. By default, the minimum and maximum memory values are 100 MB and 200 MB respectively (Xms100m -Xmx200m). Better to increase these limits to 500 MB and 1000 MB respectively as shown below.

With these settings, we can consider flume installation as completed.

Verify Apache Flume Installation:

We can verify the flume installation with $ flume-ng –help command on terminal. If we get output similar to below then flume installation is successful.

In the next posts in this category, we will discuss about setting up of various Flume agents.