org.apache.flume.sink.hdfs

Class HDFSEventSink

java.lang.Object

org.apache.flume.sink.AbstractSink

org.apache.flume.sink.hdfs.HDFSEventSink

All Implemented Interfaces:

BatchSizeSupported, Configurable, LifecycleAware, NamedComponent, Sink

public class HDFSEventSink
extends AbstractSink
implements Configurable, BatchSizeSupported

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	HDFSEventSink.WriterCallback

Nested classes/interfaces inherited from interface org.apache.flume.Sink

Sink.Status

Field Summary

Fields

Modifier and Type	Field and Description
static String	IN_USE_SUFFIX_PARAM_NAME

Constructor Summary

Constructors

Constructor and Description
HDFSEventSink()
HDFSEventSink(HDFSWriterFactory writerFactory)

Method Summary

Modifier and Type	Method and Description
void	configure(Context context) Request the implementing class to (re)configure itself.
long	getBatchSize() Returns the batch size
Sink.Status	process() Pull events out of channel and send it to HDFS.
void	start() Starts a service or component.
void	stop() Stops a service or component.
String	toString()

Methods inherited from class org.apache.flume.sink.AbstractSink

getChannel, getLifecycleState, getName, setChannel, setName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

IN_USE_SUFFIX_PARAM_NAME

public static final String IN_USE_SUFFIX_PARAM_NAME

See Also:

Constant Field Values

Constructor Detail

HDFSEventSink

public HDFSEventSink()

HDFSEventSink

public HDFSEventSink(HDFSWriterFactory writerFactory)

Method Detail

configure

public void configure(Context context)

Description copied from interface: Configurable

Request the implementing class to (re)configure itself.

When configuration parameters are changed, they must be reflected by the component asap.

There are no thread safety guarantees on when configure might be called.

Specified by:

configure in interface Configurable

process

public Sink.Status process()
                    throws EventDeliveryException

Pull events out of channel and send it to HDFS. Take at most batchSize events per Transaction. Find the corresponding bucket for the event. Ensure the file is open. Serialize the data and write it to the file on HDFS.
This method is not thread safe.

Specified by:

process in interface Sink

Returns:

READY if 1 or more Events were successfully delivered, BACKOFF if no data could be retrieved from the channel feeding this sink

Throws:

EventDeliveryException - In case of any kind of failure to deliver data to the next hop destination.

stop

public void stop()

Description copied from interface: LifecycleAware

Stops a service or component.

Implementations should determine the result of any stop logic and effect the return value of LifecycleAware.getLifecycleState() accordingly.

Specified by:

stop in interface LifecycleAware

Overrides:

stop in class AbstractSink

start

public void start()

Description copied from interface: LifecycleAware

Starts a service or component.

Implementations should determine the result of any start logic and effect the return value of LifecycleAware.getLifecycleState() accordingly.

Specified by:

start in interface LifecycleAware

Overrides:

start in class AbstractSink

toString

public String toString()

Overrides:

toString in class AbstractSink

getBatchSize

public long getBatchSize()

Description copied from interface: BatchSizeSupported

Returns the batch size

Specified by:

getBatchSize in interface BatchSizeSupported