org.apache.hadoop.hbase.wal

Class WALSplitter.OutputSink

java.lang.Object

org.apache.hadoop.hbase.wal.WALSplitter.OutputSink

Enclosing class:

WALSplitter

public abstract static class WALSplitter.OutputSink
extends Object

The following class is an abstraction class to provide a common interface to support different ways of consuming recovered edits.

Field Summary

Fields

Modifier and Type	Field and Description
protected Set<byte[]>	blacklistedRegions
protected boolean	closeAndCleanCompleted
protected WALSplitter.PipelineController	controller
protected WALSplitter.EntryBuffers	entryBuffers
protected int	numThreads
protected ConcurrentHashMap<String,Long>	regionMaximumEditLogSeqNum
protected CancelableProgressable	reporter
protected AtomicLong	skippedEdits
protected List<org.apache.hadoop.fs.Path>	splits
protected ConcurrentHashMap<String,WALSplitter.SinkWriter>	writers
protected boolean	writersClosed
protected List<WALSplitter.WriterThread>	writerThreads

Constructor Summary

Constructors

Constructor and Description
OutputSink(WALSplitter.PipelineController controller, WALSplitter.EntryBuffers entryBuffers, int numWriters)

Method Summary

Modifier and Type	Method and Description
abstract void	append(WALSplitter.RegionEntryBuffer buffer)
protected boolean	finishWriting(boolean interrupt) Wait for writer threads to dump all info to the sink
abstract List<org.apache.hadoop.fs.Path>	finishWritingAndClose()
boolean	flush() WriterThread call this function to help flush internal remaining edits in buffer before close
abstract int	getNumberOfRecoveredRegions()
abstract Map<byte[],Long>	getOutputCounts()
abstract boolean	keepRegionEvent(WAL.Entry entry) Some WALEdit's contain only KV's for account on what happened to a region.
void	startWriterThreads() Start the threads that will pump data from the entryBuffers to the output files.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

controller

protected WALSplitter.PipelineController controller

entryBuffers

protected WALSplitter.EntryBuffers entryBuffers

writers

protected ConcurrentHashMap<String,WALSplitter.SinkWriter> writers

regionMaximumEditLogSeqNum

protected final ConcurrentHashMap<String,Long> regionMaximumEditLogSeqNum

writerThreads

protected final List<WALSplitter.WriterThread> writerThreads

blacklistedRegions

protected final Set<byte[]> blacklistedRegions

closeAndCleanCompleted

protected boolean closeAndCleanCompleted

writersClosed

protected boolean writersClosed

numThreads

protected final int numThreads

reporter

protected CancelableProgressable reporter

skippedEdits

protected AtomicLong skippedEdits

splits

protected List<org.apache.hadoop.fs.Path> splits

Constructor Detail

OutputSink

public OutputSink(WALSplitter.PipelineController controller,
                  WALSplitter.EntryBuffers entryBuffers,
                  int numWriters)

Method Detail

startWriterThreads

public void startWriterThreads()

Start the threads that will pump data from the entryBuffers to the output files.

finishWriting

protected boolean finishWriting(boolean interrupt)
                         throws IOException

Wait for writer threads to dump all info to the sink

Returns:

true when there is no error

Throws:

IOException

finishWritingAndClose

public abstract List<org.apache.hadoop.fs.Path> finishWritingAndClose()
                                                               throws IOException

Throws:

IOException

getOutputCounts

public abstract Map<byte[],Long> getOutputCounts()

Returns:

a map from encoded region ID to the number of edits written out for that region.

getNumberOfRecoveredRegions

public abstract int getNumberOfRecoveredRegions()

Returns:

number of regions we've recovered

append

public abstract void append(WALSplitter.RegionEntryBuffer buffer)
                     throws IOException

Parameters:

buffer - A WAL Edit Entry

Throws:

IOException

flush

public boolean flush()
              throws IOException

WriterThread call this function to help flush internal remaining edits in buffer before close

Returns:

true when underlying sink has something to flush

Throws:

IOException

keepRegionEvent

public abstract boolean keepRegionEvent(WAL.Entry entry)

Some WALEdit's contain only KV's for account on what happened to a region. Not all sinks will want to get all of those edits.

Returns:

Return true if this sink wants to accept this region-level WALEdit.