HFileBlock (Apache HBase 2.1.0-cdh6.3.2 API)

java.lang.Object
- org.apache.hadoop.hbase.io.hfile.HFileBlock

All Implemented Interfaces:

HeapSize, Cacheable
```
@InterfaceAudience.Private
public class HFileBlock
extends Object
implements Cacheable
```
Cacheable Blocks of an HFile version 2 file. Version 2 was introduced in hbase-0.92.0.
Version 1 was the original file block. Version 2 was introduced when we changed the hbase file format to support multi-level block indexes and compound bloom filters (HBASE-3857). Support for Version 1 was removed in hbase-1.3.0.
HFileBlock: Version 2
In version 2, a block is structured as follows:
- Header: See Writer#putHeader() for where header is written; header total size is HFILEBLOCK_HEADER_SIZE
  - 0. blockType: Magic record identifying the BlockType (8 bytes): e.g. DATABLK*
  - 1. onDiskSizeWithoutHeader: Compressed -- a.k.a 'on disk' -- block size, excluding header, but including tailing checksum bytes (4 bytes)
  - 2. uncompressedSizeWithoutHeader: Uncompressed block size, excluding header, and excluding checksum bytes (4 bytes)
  - 3. prevBlockOffset: The offset of the previous block of the same type (8 bytes). This is used to navigate to the previous block without having to go to the block index
  - 4: For minorVersions >=1, the ordinal describing checksum type (1 byte)
  - 5: For minorVersions >=1, the number of data bytes/checksum chunk (4 bytes)
  - 6: onDiskDataSizeWithHeader: For minorVersions >=1, the size of data 'on disk', including header, excluding checksums (4 bytes)
- Raw/Compressed/Encrypted/Encoded data: The compression algorithm is the same for all the blocks in an HFile. If compression is NONE, this is just raw, serialized Cells.
- Tail: For minorVersions >=1, a series of 4 byte checksums, one each for the number of bytes specified by bytesPerChecksum.
Caching
Caches cache whole blocks with trailing checksums if any. We then tag on some metadata, the content of BLOCK_METADATA_SPACE which will be flag on if we are doing 'hbase' checksums and then the offset into the file which is needed when we re-make a cache key when we return the block to the cache as 'done'. See Cacheable.serialize(ByteBuffer, boolean) and Cacheable.getDeserializer().
TODO: Should we cache the checksums? Down in Writer#getBlockForCaching(CacheConfig) where we make a block to cache-on-write, there is an attempt at turning off checksums. This is not the only place we get blocks to cache. We also will cache the raw return from an hdfs read. In this case, the checksums may be present. If the cache is backed by something that doesn't do ECC, say an SSD, we might want to preserve checksums. For now this is open question.
TODO: Over in BucketCache, we save a block allocation by doing a custom serialization. Be sure to change it if serialization changes in here. Could we add a method here that takes an IOEngine and that then serializes to it rather than expose our internals over in BucketCache? IOEngine is in the bucket subpackage. Pull it up? Then this class knows about bucketcache. Ugh.

Nested Class Summary
- Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.hfile.Cacheable
  Cacheable.MemoryType

Field Summary

Fields
Modifier and Type Field and Description

static boolean DONT_FILL_HEADER

static boolean FILL_HEADER

static int MULTI_BYTE_BUFFER_HEAP_SIZE

Fields
Modifier and Type	Field and Description
`static boolean`	`DONT_FILL_HEADER`
`static boolean`	`FILL_HEADER`
`static int`	`MULTI_BYTE_BUFFER_HEAP_SIZE`

Constructor Summary

Constructors
Constructor and Description
`HFileBlock(BlockType blockType, int onDiskSizeWithoutHeader, int uncompressedSizeWithoutHeader, long prevBlockOffset, ByteBuffer b, boolean fillHeader, long offset, int nextBlockOnDiskSize, int onDiskDataSizeWithHeader, HFileContext fileContext)` Creates a new `HFile` block from the given fields.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`HFileBlock`	`deepClone()`
`boolean`	`equals(Object comparison)`
`BlockType`	`getBlockType()`
`ByteBuff`	`getBufferReadOnly()` Returns a read-only duplicate of the buffer this block stores internally ready to be read.
`ByteBuff`	`getBufferWithoutHeader()` Returns a buffer that does not include the header or checksum.
`CacheableDeserializer<Cacheable>`	`getDeserializer()` Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.
`Cacheable.MemoryType`	`getMemoryType()`
`ByteBuffer`	`getMetaData()` For use by bucketcache.
`int`	`getOnDiskSizeWithHeader()`
`int`	`getSerializedLength()` Returns the length of the ByteBuffer required to serialized the object.
`int`	`hashCode()`
`int`	`headerSize()` Returns the size of this block header.
`static int`	`headerSize(boolean usesHBaseChecksum)` Maps a minor version to the size of the header.
`long`	`heapSize()`
`boolean`	`isUnpacked()` Return true when this block's buffer has been unpacked, false otherwise.
`void`	`serialize(ByteBuffer destination, boolean includeNextBlockMetadata)` Serializes its data into destination.
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

- Field Detail
  - FILL_HEADER
```
public static final boolean FILL_HEADER
```
    See Also:
    
    Constant Field Values
  - DONT_FILL_HEADER
```
public static final boolean DONT_FILL_HEADER
```
    See Also:
    
    Constant Field Values
  - MULTI_BYTE_BUFFER_HEAP_SIZE
```
public static final int MULTI_BYTE_BUFFER_HEAP_SIZE
```
- Constructor Detail
  - HFileBlock
```
public HFileBlock(BlockType blockType,
                  int onDiskSizeWithoutHeader,
                  int uncompressedSizeWithoutHeader,
                  long prevBlockOffset,
                  ByteBuffer b,
                  boolean fillHeader,
                  long offset,
                  int nextBlockOnDiskSize,
                  int onDiskDataSizeWithHeader,
                  HFileContext fileContext)
```
    Creates a new HFile block from the given fields. This constructor is used only while writing blocks and caching, and is sitting in a byte buffer and we want to stuff the block into cache. See Writer#getBlockForCaching(CacheConfig).
    TODO: The caller presumes no checksumming required of this block instance since going into cache; checksum already verified on underlying block data pulled in from filesystem. Is that correct? What if cache is SSD?
    
    Parameters:
    
    blockType - the type of this block, see BlockType
    
    onDiskSizeWithoutHeader - see onDiskSizeWithoutHeader
    
    uncompressedSizeWithoutHeader - see uncompressedSizeWithoutHeader
    
    prevBlockOffset - see prevBlockOffset
    
    b - block header (HConstants.HFILEBLOCK_HEADER_SIZE bytes)
    
    fillHeader - when true, write the first 4 header fields into passed buffer.
    
    offset - the file offset the block was read from
    
    onDiskDataSizeWithHeader - see onDiskDataSizeWithHeader
    
    fileContext - HFile meta data
- Method Detail
  - getBlockType
```
public BlockType getBlockType()
```
    Specified by:
    
    getBlockType in interface Cacheable
    
    Returns:
    
    the block type of this cached HFile block
  - getOnDiskSizeWithHeader
```
public int getOnDiskSizeWithHeader()
```
    Returns:
    
    the on-disk size of header + data part + checksum.
  - getBufferWithoutHeader
```
public ByteBuff getBufferWithoutHeader()
```
    Returns a buffer that does not include the header or checksum.
    
    Returns:
    
    the buffer with header skipped and checksum omitted.
  - getBufferReadOnly
```
public ByteBuff getBufferReadOnly()
```
    Returns a read-only duplicate of the buffer this block stores internally ready to be read. Clients must not modify the buffer object though they may set position and limit on the returned buffer since we pass back a duplicate. This method has to be public because it is used in CompoundBloomFilter to avoid object creation on every Bloom filter lookup, but has to be used with caution. Buffer holds header, block content, and any follow-on checksums if present.
    
    Returns:
    
    the buffer of this block for read-only operations
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - isUnpacked
```
public boolean isUnpacked()
```
    Return true when this block's buffer has been unpacked, false otherwise. Note this is a calculated heuristic, not tracked attribute of the block.
  - heapSize
```
public long heapSize()
```
    Specified by:
    
    heapSize in interface HeapSize
    
    Returns:
    
    Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.
  - getSerializedLength
```
public int getSerializedLength()
```
    Description copied from interface: Cacheable
    
    Returns the length of the ByteBuffer required to serialized the object. If the object cannot be serialized, it should return 0.
    
    Specified by:
    
    getSerializedLength in interface Cacheable
    
    Returns:
    
    int length in bytes of the serialized form or 0 if the object cannot be cached.
  - serialize
```
public void serialize(ByteBuffer destination,
                      boolean includeNextBlockMetadata)
```
    Description copied from interface: Cacheable
    
    Serializes its data into destination.
    
    Specified by:
    
    serialize in interface Cacheable
    
    Parameters:
    
    destination - Where to serialize to
    
    includeNextBlockMetadata - Whether to include nextBlockMetadata in the Cache block.
  - getMetaData
```
public ByteBuffer getMetaData()
```
    For use by bucketcache. This exposes internals.
  - getDeserializer
```
public CacheableDeserializer<Cacheable> getDeserializer()
```
    Description copied from interface: Cacheable
    
    Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.
    
    Specified by:
    
    getDeserializer in interface Cacheable
    
    Returns:
    
    CacheableDeserialzer instance.
  - hashCode
```
public int hashCode()
```
    Overrides:
    
    hashCode in class Object
  - equals
```
public boolean equals(Object comparison)
```
    Overrides:
    
    equals in class Object
  - headerSize
```
public int headerSize()
```
    Returns the size of this block header.
  - headerSize
```
public static int headerSize(boolean usesHBaseChecksum)
```
    Maps a minor version to the size of the header.
  - getMemoryType
```
public Cacheable.MemoryType getMemoryType()
```
    Specified by:
    
    getMemoryType in interface Cacheable
    
    Returns:
    
    the MemoryType of this Cacheable
  - deepClone
```
public HFileBlock deepClone()
```

Class HFileBlock

HFileBlock: Version 2

Caching

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.hfile.Cacheable

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

FILL_HEADER

DONT_FILL_HEADER

MULTI_BYTE_BUFFER_HEAP_SIZE

Constructor Detail

HFileBlock

Method Detail

getBlockType

getOnDiskSizeWithHeader

getBufferWithoutHeader

getBufferReadOnly

toString

isUnpacked

heapSize

getSerializedLength

serialize

getMetaData

getDeserializer

hashCode

equals

headerSize

headerSize

getMemoryType

deepClone