org.apache.hadoop.hbase.regionserver

Class HMobStore

java.lang.Object

org.apache.hadoop.hbase.regionserver.HStore

org.apache.hadoop.hbase.regionserver.HMobStore

All Implemented Interfaces:

ConfigurationObserver, PropagatingConfigurationObserver, HeapSize, Store, StoreConfigInformation

@InterfaceAudience.Private
public class HMobStore
extends HStore

The store implementation to save MOBs (medium objects), it extends the HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek to HBase is done. The store implements how we save MOBs by extending HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.HStore

BLOCK_STORAGE_POLICY_KEY, BLOCKING_STOREFILES_KEY, blocksize, bytesPerChecksum, cacheConf, checksumType, COMPACTCHECKER_INTERVAL_MULTIPLIER_KEY, comparator, conf, cryptoContext, DEEP_OVERHEAD, DEFAULT_BLOCK_STORAGE_POLICY, DEFAULT_BLOCKING_STOREFILE_COUNT, DEFAULT_COMPACTCHECKER_INTERVAL_MULTIPLIER, FIXED_OVERHEAD, memstore, MEMSTORE_CLASS_NAME, region

Fields inherited from interface org.apache.hadoop.hbase.regionserver.Store

NO_PRIORITY, PRIORITY_USER

Constructor Summary

Constructors

Constructor and Description
HMobStore(HRegion region, ColumnFamilyDescriptor family, org.apache.hadoop.conf.Configuration confParam)

Method Summary

Modifier and Type	Method and Description
void	commitFile(org.apache.hadoop.fs.Path sourceFile, org.apache.hadoop.fs.Path targetPath) Commits the mob file.
protected void	createCacheConf(ColumnFamilyDescriptor family) Creates the mob cache config.
StoreFileWriter	createDelFileWriterInTmp(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey) Creates the writer for the del file in temp directory.
protected KeyValueScanner	createScanner(Scan scan, ScanInfo scanInfo, NavigableSet<byte[]> targetCols, long readPt) Gets the MobStoreScanner or MobReversedStoreScanner.
protected StoreEngine<?,?,?,?>	createStoreEngine(HStore store, org.apache.hadoop.conf.Configuration conf, CellComparator cellComparator) Creates the mob store engine.
StoreFileWriter	createWriterInTmp(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction) Creates the writer for the mob file in temp directory.
StoreFileWriter	createWriterInTmp(MobFileName mobFileName, org.apache.hadoop.fs.Path basePath, long maxKeyCount, Compression.Algorithm compression, boolean isCompaction) Creates the writer for the mob file in temp directory.
StoreFileWriter	createWriterInTmp(String date, org.apache.hadoop.fs.Path basePath, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction) Creates the writer for the mob file in temp directory.
long	getCellsCountCompactedFromMob()
long	getCellsCountCompactedToMob()
long	getCellsSizeCompactedFromMob()
long	getCellsSizeCompactedToMob()
org.apache.hadoop.conf.Configuration	getConfiguration() Gets current config.
long	getMobFlushCount()
long	getMobFlushedCellsCount()
long	getMobFlushedCellsSize()
long	getMobScanCellsCount()
long	getMobScanCellsSize()
org.apache.hadoop.fs.Path	getPath() Gets the mob file path.
byte[]	getRefCellTags()
Cell	resolve(Cell reference, boolean cacheBlocks) Reads the cell from the mob file, and the read point does not count.
Cell	resolve(Cell reference, boolean cacheBlocks, long readPt, boolean readEmptyValueOnMobCellMiss) Reads the cell from the mob file.
void	updateCellsCountCompactedFromMob(long count)
void	updateCellsCountCompactedToMob(long count)
void	updateCellsSizeCompactedFromMob(long size)
void	updateCellsSizeCompactedToMob(long size)
void	updateMobFlushCount()
void	updateMobFlushedCellsCount(long count)
void	updateMobFlushedCellsSize(long size)
void	updateMobScanCellsCount(long count)
void	updateMobScanCellsSize(long size)

Methods inherited from class org.apache.hadoop.hbase.regionserver.HStore

add, add, addChangedReaderObserver, areWritesEnabled, assertBulkLoadHFileOk, bulkLoadHFile, bulkLoadHFile, cancelRequestedCompaction, canSplit, close, closeAndArchiveCompactedFiles, closeAndArchiveCompactedFiles, compact, compactRecentForTestingAssumingDefaultPolicy, completeCompaction, createFlushContext, createStoreFileAndReader, createWriterInTmp, deleteChangedReaderObserver, deregisterChildren, determineTTLFromFamily, doCompaction, flushCache, getAvgStoreFileAge, getBlockingFileCount, getBytesPerChecksum, getCacheConfig, getChecksumType, getCloseCheckInterval, getColumnFamilyDescriptor, getColumnFamilyName, getCompactedCellsCount, getCompactedCellsSize, getCompactedFiles, getCompactedFilesCount, getCompactionCheckMultiplier, getCompactionPressure, getCompactionProgress, getCompactPriority, getComparator, getCoprocessorHost, getCurrentParallelPutCount, getDataBlockEncoder, getFileSystem, getFlushableSize, getFlushedCellsCount, getFlushedCellsSize, getFlushedOutputFileSize, getHFilesSize, getHRegion, getLastCompactSize, getMajorCompactedCellsCount, getMajorCompactedCellsSize, getMaxMemStoreTS, getMaxSequenceId, getMaxStoreFileAge, getMemStoreFlushSize, getMemStoreSize, getMinStoreFileAge, getNumHFiles, getNumReferenceFiles, getOffPeakHours, getRegionFileSystem, getRegionInfo, getScanInfo, getScanner, getScanners, getScanners, getScanners, getScanners, getSize, getSmallestReadPoint, getSnapshotSize, getSplitPoint, getStoreEngine, getStorefiles, getStorefilesCount, getStorefilesRootLevelIndexSize, getStorefilesSize, getStoreFileTtl, getStoreHomedir, getStoreHomedir, getStoreSizeUncompressed, getTableName, getTotalStaticBloomSize, getTotalStaticIndexSize, hasReferences, hasTooManyStoreFiles, heapSize, isPrimaryReplicaStore, isSloppyMemStore, needsCompaction, onConfigurationChange, postSnapshotOperation, preBulkLoadHFile, preFlushSeqIDEstimation, preSnapshotOperation, recreateScanners, refreshStoreFiles, refreshStoreFiles, registerChildren, replayCompactionMarker, requestCompaction, requestCompaction, shouldPerformMajorCompaction, startReplayingFromWAL, stopReplayingFromWAL, throttleCompaction, timeOfOldestEdit, toString, triggerMajorCompaction, upsert

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HMobStore

public HMobStore(HRegion region,
                 ColumnFamilyDescriptor family,
                 org.apache.hadoop.conf.Configuration confParam)
          throws IOException

Throws:

IOException

Method Detail

createCacheConf

protected void createCacheConf(ColumnFamilyDescriptor family)

Creates the mob cache config.

Overrides:

createCacheConf in class HStore

Parameters:

family - The current column family.

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

Gets current config.

createScanner

protected KeyValueScanner createScanner(Scan scan,
                                        ScanInfo scanInfo,
                                        NavigableSet<byte[]> targetCols,
                                        long readPt)
                                 throws IOException

Gets the MobStoreScanner or MobReversedStoreScanner. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.

Overrides:

createScanner in class HStore

Throws:

IOException

createStoreEngine

protected StoreEngine<?,?,?,?> createStoreEngine(HStore store,
                                                 org.apache.hadoop.conf.Configuration conf,
                                                 CellComparator cellComparator)
                                          throws IOException

Creates the mob store engine.

Overrides:

createStoreEngine in class HStore

Parameters:

store - The store. An unfortunate dependency needed due to it being passed to coprocessors via the compactor.

conf - Store configuration.

cellComparator - KVComparator for storeFileManager.

Returns:

StoreEngine to use.

Throws:

IOException

createWriterInTmp

public StoreFileWriter createWriterInTmp(Date date,
                                         long maxKeyCount,
                                         Compression.Algorithm compression,
                                         byte[] startKey,
                                         boolean isCompaction)
                                  throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

date - The latest date of written cells.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

isCompaction - If the writer is used in compaction.

Returns:

The writer for the mob file.

Throws:

IOException

createDelFileWriterInTmp

public StoreFileWriter createDelFileWriterInTmp(Date date,
                                                long maxKeyCount,
                                                Compression.Algorithm compression,
                                                byte[] startKey)
                                         throws IOException

Creates the writer for the del file in temp directory. The del file keeps tracking the delete markers. Its name has a suffix _del, the format is [0-9a-f]+(_del)?.

Parameters:

date - The latest date of written cells.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

Returns:

The writer for the del file.

Throws:

IOException

createWriterInTmp

public StoreFileWriter createWriterInTmp(String date,
                                         org.apache.hadoop.fs.Path basePath,
                                         long maxKeyCount,
                                         Compression.Algorithm compression,
                                         byte[] startKey,
                                         boolean isCompaction)
                                  throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

date - The date string, its format is yyyymmmdd.

basePath - The basic path for a temp directory.

maxKeyCount - The key count.

compression - The compression algorithm.

startKey - The start key.

isCompaction - If the writer is used in compaction.

Returns:

The writer for the mob file.

Throws:

IOException

createWriterInTmp

public StoreFileWriter createWriterInTmp(MobFileName mobFileName,
                                         org.apache.hadoop.fs.Path basePath,
                                         long maxKeyCount,
                                         Compression.Algorithm compression,
                                         boolean isCompaction)
                                  throws IOException

Creates the writer for the mob file in temp directory.

Parameters:

mobFileName - The mob file name.

basePath - The basic path for a temp directory.

maxKeyCount - The key count.

compression - The compression algorithm.

isCompaction - If the writer is used in compaction.

Returns:

The writer for the mob file.

Throws:

IOException

commitFile

public void commitFile(org.apache.hadoop.fs.Path sourceFile,
                       org.apache.hadoop.fs.Path targetPath)
                throws IOException

Commits the mob file.

Parameters:

sourceFile - The source file.

targetPath - The directory path where the source file is renamed to.

Throws:

IOException

resolve

public Cell resolve(Cell reference,
                    boolean cacheBlocks)
             throws IOException

Reads the cell from the mob file, and the read point does not count. This is used for DefaultMobStoreCompactor where we can read empty value for the missing cell.

Parameters:

reference - The cell found in the HBase, its value is a path to a mob file.

cacheBlocks - Whether the scanner should cache blocks.

Returns:

The cell found in the mob file.

Throws:

IOException

resolve

public Cell resolve(Cell reference,
                    boolean cacheBlocks,
                    long readPt,
                    boolean readEmptyValueOnMobCellMiss)
             throws IOException

Reads the cell from the mob file.

Parameters:

reference - The cell found in the HBase, its value is a path to a mob file.

cacheBlocks - Whether the scanner should cache blocks.

readPt - the read point.

readEmptyValueOnMobCellMiss - Whether return null value when the mob file is missing or corrupt.

Returns:

The cell found in the mob file.

Throws:

IOException

getPath

public org.apache.hadoop.fs.Path getPath()

Gets the mob file path.

Returns:

The mob file path.

updateCellsCountCompactedToMob

public void updateCellsCountCompactedToMob(long count)

getCellsCountCompactedToMob

public long getCellsCountCompactedToMob()

updateCellsCountCompactedFromMob

public void updateCellsCountCompactedFromMob(long count)

getCellsCountCompactedFromMob

public long getCellsCountCompactedFromMob()

updateCellsSizeCompactedToMob

public void updateCellsSizeCompactedToMob(long size)

getCellsSizeCompactedToMob

public long getCellsSizeCompactedToMob()

updateCellsSizeCompactedFromMob

public void updateCellsSizeCompactedFromMob(long size)

getCellsSizeCompactedFromMob

public long getCellsSizeCompactedFromMob()

updateMobFlushCount

public void updateMobFlushCount()

getMobFlushCount

public long getMobFlushCount()

updateMobFlushedCellsCount

public void updateMobFlushedCellsCount(long count)

getMobFlushedCellsCount

public long getMobFlushedCellsCount()

updateMobFlushedCellsSize

public void updateMobFlushedCellsSize(long size)

getMobFlushedCellsSize

public long getMobFlushedCellsSize()

updateMobScanCellsCount

public void updateMobScanCellsCount(long count)

getMobScanCellsCount

public long getMobScanCellsCount()

updateMobScanCellsSize

public void updateMobScanCellsSize(long size)

getMobScanCellsSize

public long getMobScanCellsSize()

getRefCellTags

public byte[] getRefCellTags()