Table of Contents
org.apache.flume.EventDeliveryException: Failed to open file
We will receive this error message when a flume agent is started and it is trying to start FILE_ROLL sink with a given target sink directory. Below are the error messages sequence from ~/logs/flume.log file.
21 Sep 2014 15:16:59,297 INFO [lifecycleSupervisor-1-5] (org.apache.flume.instrumentation.MonitoredCounterGroup.register:119) - Monitored counter group for type: SOURCE, name: exec-source: Successfully registered new MBean.
21 Sep 2014 15:16:59,310 INFO [lifecycleSupervisor-1-5] (org.apache.flume.instrumentation.MonitoredCounterGroup.start:95) - Component type: SOURCE, name: exec-source started
21 Sep 2014 15:17:04,310 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to open file $HOME/flume/files/1411292818271-1 while delivering event
Caused by: java.io.FileNotFoundException: /home/hadoop/flume/files/1411292818271-1 (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
... 3 more
When a flume agent is started with FILE_ROLL sink type and without creating <sink.directory> folder prior to starting agent or
Flume user doesn’t have enough permissions (write access) to <sink.directory> to create files under it, then we will receive this error message.
In our above error scenario, the directory /home/hadoop/flume/files is not created prior to the start of agent Agent2. So, we have received the above error message.
If we didn’t have the destination directory already created but flume user has write access to create the folder, even then we will receive the above error message because Flume JVM process will not create the <sink.directory> on the fly while running Flume agent.
Parent directory given in AgentX.sinks.XXXXX.sink.directory property should already be created before starting flume agent and the flume user should have write access to it.
Even if the flume user has access to create files and parent directory is not created prior to starting agent,flume process will not create the directory on the fly.