org.apache.hadoop.hbase.regionserver.wal

Class ProtobufLogReader

java.lang.Object

org.apache.hadoop.hbase.regionserver.wal.ReaderBase

org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader

All Implemented Interfaces:

Closeable, AutoCloseable, AbstractFSWALProvider.Reader, WAL.Reader

Direct Known Subclasses:

SecureProtobufLogReader

@InterfaceAudience.LimitedPrivate(value={"Coprocesssor","Phoenix","Configuration"})
public class ProtobufLogReader
extends ReaderBase

A Protobuf based WAL has the following structure:

<PB_WAL_MAGIC><WALHeader><WALEdits>...<WALEdits><Trailer> <TrailerSize> <PB_WAL_COMPLETE_MAGIC>

The Reader reads meta information (WAL Compression state, WALTrailer, etc) in ProtobufLogReader#initReader(FSDataInputStream). A WALTrailer is an extensible structure which is appended at the end of the WAL. This is empty for now; it can contain some meta information such as Region level stats, etc in future.

Field Summary

Fields

Modifier and Type	Field and Description
protected WALCellCodec.ByteStringUncompressor	byteStringUncompressor
protected Codec.Decoder	cellDecoder
protected boolean	hasCompression
protected boolean	hasTagCompression
protected org.apache.hadoop.fs.FSDataInputStream	inputStream
static byte[]	PB_WAL_COMPLETE_MAGIC
static byte[]	PB_WAL_MAGIC
protected WALProtos.WALTrailer	trailer
protected int	trailerWarnSize

Fields inherited from class org.apache.hadoop.hbase.regionserver.wal.ReaderBase

compressionContext, conf, edit, emptyCompressionContext, fileLength, fs, path

Constructor Summary

Constructors

Constructor and Description
ProtobufLogReader()

Method Summary

Modifier and Type	Method and Description
void	close()
protected WALCellCodec	getCodec(org.apache.hadoop.conf.Configuration conf, String cellCodecClsName, CompressionContext compressionContext)
String	getCodecClsName()
long	getPosition()
List<String>	getWriterClsNames()
protected boolean	hasCompression()
protected boolean	hasTagCompression()
void	init(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataInputStream stream)
protected void	initAfterCompression() Initializes the compression after the shared stuff has been initialized.
protected void	initAfterCompression(String cellCodecClsName) Initializes the compression after the shared stuff has been initialized.
protected String	initReader(org.apache.hadoop.fs.FSDataInputStream stream) Initializes the log reader with a particular stream (may be null).
protected org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.WALHdrContext	readHeader(WALProtos.WALHeader.Builder builder, org.apache.hadoop.fs.FSDataInputStream stream)
protected boolean	readNext(WAL.Entry entry) Read next entry.
void	reset()
protected void	seekOnFs(long pos) Performs a filesystem-level seek to a certain position in an underlying file.
long	trailerSize()

Methods inherited from class org.apache.hadoop.hbase.regionserver.wal.ReaderBase

next, next, seek

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PB_WAL_MAGIC

@InterfaceAudience.Private
public static final byte[] PB_WAL_MAGIC

PB_WAL_COMPLETE_MAGIC

@InterfaceAudience.Private
public static final byte[] PB_WAL_COMPLETE_MAGIC

inputStream

protected org.apache.hadoop.fs.FSDataInputStream inputStream

cellDecoder

protected Codec.Decoder cellDecoder

byteStringUncompressor

protected WALCellCodec.ByteStringUncompressor byteStringUncompressor

hasCompression

protected boolean hasCompression

hasTagCompression

protected boolean hasTagCompression

trailer

protected WALProtos.WALTrailer trailer

trailerWarnSize

protected int trailerWarnSize

Constructor Detail

ProtobufLogReader

public ProtobufLogReader()

Method Detail

trailerSize

@InterfaceAudience.Private
public long trailerSize()

close

public void close()
           throws IOException

Throws:

IOException

getPosition

public long getPosition()
                 throws IOException

Throws:

IOException

reset

public void reset()
           throws IOException

Throws:

IOException

init

public void init(org.apache.hadoop.fs.FileSystem fs,
                 org.apache.hadoop.fs.Path path,
                 org.apache.hadoop.conf.Configuration conf,
                 org.apache.hadoop.fs.FSDataInputStream stream)
          throws IOException

Specified by:

init in interface AbstractFSWALProvider.Reader

Overrides:

init in class ReaderBase

Parameters:

fs - File system.

path - Path.

conf - Configuration.

stream - Input stream that may have been pre-opened by the caller; may be null.

Throws:

IOException

initReader

protected String initReader(org.apache.hadoop.fs.FSDataInputStream stream)
                     throws IOException

Description copied from class: ReaderBase

Initializes the log reader with a particular stream (may be null). Reader assumes ownership of the stream if not null and may use it. Called once.

Specified by:

initReader in class ReaderBase

Returns:

the class name of cell Codec, null if such information is not available

Throws:

IOException

getWriterClsNames

public List<String> getWriterClsNames()

getCodecClsName

public String getCodecClsName()

readHeader

protected org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.WALHdrContext readHeader(WALProtos.WALHeader.Builder builder,
                                                                                              org.apache.hadoop.fs.FSDataInputStream stream)
                                                                                       throws IOException

Throws:

IOException

getCodec

protected WALCellCodec getCodec(org.apache.hadoop.conf.Configuration conf,
                                String cellCodecClsName,
                                CompressionContext compressionContext)
                         throws IOException

Throws:

IOException

initAfterCompression

protected void initAfterCompression()
                             throws IOException

Description copied from class: ReaderBase

Initializes the compression after the shared stuff has been initialized. Called once.

Specified by:

initAfterCompression in class ReaderBase

Throws:

IOException

initAfterCompression

protected void initAfterCompression(String cellCodecClsName)
                             throws IOException

Description copied from class: ReaderBase

Initializes the compression after the shared stuff has been initialized. Called once.

Specified by:

initAfterCompression in class ReaderBase

Parameters:

cellCodecClsName - class name of cell Codec

Throws:

IOException

hasCompression

protected boolean hasCompression()

Specified by:

hasCompression in class ReaderBase

Returns:

Whether compression is enabled for this log.

hasTagCompression

protected boolean hasTagCompression()

Specified by:

hasTagCompression in class ReaderBase

Returns:

Whether tag compression is enabled for this log.

readNext

protected boolean readNext(WAL.Entry entry)
                    throws IOException

Description copied from class: ReaderBase

Read next entry.

Specified by:

readNext in class ReaderBase

Parameters:

entry - The entry to read into.

Returns:

Whether there was anything to read.

Throws:

IOException

seekOnFs

protected void seekOnFs(long pos)
                 throws IOException

Description copied from class: ReaderBase

Performs a filesystem-level seek to a certain position in an underlying file.

Specified by:

seekOnFs in class ReaderBase

Throws:

IOException