org.apache.flume.sink

Class FailoverSinkProcessor

java.lang.Object

org.apache.flume.sink.AbstractSinkProcessor

org.apache.flume.sink.FailoverSinkProcessor

All Implemented Interfaces:

Configurable, LifecycleAware, SinkProcessor

public class FailoverSinkProcessor
extends AbstractSinkProcessor

FailoverSinkProcessor maintains a prioritized list of sinks, guarranteeing that so long as one is available events will be processed. The failover mechanism works by relegating failed sinks to a pool where they are assigned a cooldown period, increasing with sequential failures before they are retried. Once a sink successfully sends an event it is restored to the live pool. FailoverSinkProcessor is in no way thread safe and expects to be run via SinkRunner Additionally, setSinks must be called before configure, and additional sinks cannot be added while running To configure, set a sink groups processor to "failover" and set priorities for individual sinks, all priorities must be unique. Furthermore, an upper limit to failover time can be set(in miliseconds) using maxpenalty Ex) host1.sinkgroups = group1 host1.sinkgroups.group1.sinks = sink1 sink2 host1.sinkgroups.group1.processor.type = failover host1.sinkgroups.group1.processor.priority.sink1 = 5 host1.sinkgroups.group1.processor.priority.sink2 = 10 host1.sinkgroups.group1.processor.maxpenalty = 10000

Constructor Summary

Constructors

Constructor and Description
FailoverSinkProcessor()

Method Summary

Modifier and Type	Method and Description
void	configure(Context context) Request the implementing class to (re)configure itself.
Sink.Status	process() Handle a request to poll the owned sinks.
void	setSinks(List<Sink> sinks) Set all sinks to work with.

Methods inherited from class org.apache.flume.sink.AbstractSinkProcessor

getLifecycleState, getSinks, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FailoverSinkProcessor

public FailoverSinkProcessor()

Method Detail

configure

public void configure(Context context)

Description copied from interface: Configurable

Request the implementing class to (re)configure itself.

When configuration parameters are changed, they must be reflected by the component asap.

There are no thread safety guarantees on when configure might be called.

process

public Sink.Status process()
                    throws EventDeliveryException

Description copied from interface: SinkProcessor

Handle a request to poll the owned sinks.

The processor is expected to call Sink.process() on whatever sink(s) appropriate, handling failures as appropriate and throwing EventDeliveryException when there is a failure to deliver any events according to the delivery policy defined by the sink processor implementation. See specific implementations of this interface for delivery behavior and policies.

Returns:

Returns READY if events were successfully consumed, or BACKOFF if no events were available in the channel to consume.

Throws:

EventDeliveryException - if the behavior guaranteed by the processor couldn't be carried out.

setSinks

public void setSinks(List<Sink> sinks)

Description copied from interface: SinkProcessor

Set all sinks to work with.

Sink specific parameters are passed to the processor via configure

Specified by:

setSinks in interface SinkProcessor

Overrides:

setSinks in class AbstractSinkProcessor

Parameters:

sinks - A non-null, non-empty list of sinks to be chosen from by the processor