@InterfaceAudience.Private public class CacheConfig extends Object
Modifier and Type | Field and Description |
---|---|
static String |
BLOCKCACHE_BLOCKSIZE_KEY
The target block size used by blockcache instances.
|
static String |
BUCKET_CACHE_BUCKETS_KEY
A comma-delimited array of values for use as bucket sizes.
|
static String |
BUCKET_CACHE_PERSISTENT_PATH_KEY
If the chosen ioengine can persist its state across restarts, the path to the file to persist
to.
|
static String |
BUCKET_CACHE_WRITER_QUEUE_KEY |
static String |
BUCKET_CACHE_WRITER_THREADS_KEY |
static String |
CACHE_BLOCKS_ON_WRITE_KEY
Configuration key to cache data blocks on write.
|
static String |
CACHE_BLOOM_BLOCKS_ON_WRITE_KEY
Configuration key to cache compound bloom filter blocks on write.
|
static String |
CACHE_DATA_BLOCKS_COMPRESSED_KEY
Configuration key to cache data blocks in compressed and/or encrypted format.
|
static String |
CACHE_DATA_ON_READ_KEY
Configuration key to cache data blocks on read.
|
static String |
CACHE_INDEX_BLOCKS_ON_WRITE_KEY
Configuration key to cache leaf and intermediate-level index blocks on
write.
|
static int |
DEFAULT_BUCKET_CACHE_WRITER_QUEUE |
static int |
DEFAULT_BUCKET_CACHE_WRITER_THREADS
Defaults for Bucket cache
|
static boolean |
DEFAULT_CACHE_BLOOMS_ON_WRITE |
static boolean |
DEFAULT_CACHE_DATA_COMPRESSED |
static boolean |
DEFAULT_CACHE_DATA_ON_READ |
static boolean |
DEFAULT_CACHE_DATA_ON_WRITE |
static boolean |
DEFAULT_CACHE_INDEXES_ON_WRITE |
static boolean |
DEFAULT_EVICT_ON_CLOSE |
static boolean |
DEFAULT_IN_MEMORY |
static boolean |
DEFAULT_PREFETCH_ON_OPEN |
static CacheConfig |
DISABLED
Disabled cache configuration
|
static String |
EVICT_BLOCKS_ON_CLOSE_KEY
Configuration key to evict all blocks of a given file from the block cache
when the file is closed.
|
static String |
PREFETCH_BLOCKS_ON_OPEN_KEY
Configuration key to prefetch all blocks of a given file into the block cache
when the file is opened.
|
Constructor and Description |
---|
CacheConfig(CacheConfig cacheConf)
Constructs a cache configuration copied from the specified configuration.
|
CacheConfig(org.apache.hadoop.conf.Configuration conf)
Create a cache configuration using the specified configuration object and
defaults for family level settings.
|
CacheConfig(org.apache.hadoop.conf.Configuration conf,
ColumnFamilyDescriptor family)
Create a cache configuration using the specified configuration object and
family descriptor.
|
Modifier and Type | Method and Description |
---|---|
BlockCache |
getBlockCache()
Returns the block cache.
|
CacheStats |
getL2CacheStats() |
static LruBlockCache |
getOnHeapCache(org.apache.hadoop.conf.Configuration c) |
CacheStats |
getOnHeapCacheStats() |
static BlockCache |
instantiateBlockCache(org.apache.hadoop.conf.Configuration conf)
Returns the block cache or
null in case none should be used. |
boolean |
isBlockCacheEnabled()
Checks whether the block cache is enabled.
|
boolean |
isInMemory() |
void |
setCacheDataOnWrite(boolean cacheDataOnWrite)
Only used for testing.
|
void |
setEvictOnClose(boolean evictOnClose)
Only used for testing.
|
boolean |
shouldCacheBlockOnRead(BlockType.BlockCategory category)
Should we cache a block of a particular category? We always cache
important blocks such as index blocks, as long as the block cache is
available.
|
boolean |
shouldCacheBloomsOnWrite() |
boolean |
shouldCacheCompressed(BlockType.BlockCategory category) |
boolean |
shouldCacheDataCompressed() |
boolean |
shouldCacheDataOnRead()
Returns whether the DATA blocks of this HFile should be cached on read or not (we always
cache the meta blocks, the INDEX and BLOOM blocks).
|
boolean |
shouldCacheDataOnWrite() |
boolean |
shouldCacheIndexesOnWrite() |
boolean |
shouldDropBehindCompaction() |
boolean |
shouldEvictOnClose() |
boolean |
shouldLockOnCacheMiss(BlockType blockType)
If we make sure the block could not be cached, we will not acquire the lock
otherwise we will acquire lock
|
boolean |
shouldPrefetchOnOpen() |
boolean |
shouldReadBlockFromCache(BlockType blockType)
Return true if we may find this type of block in block cache.
|
String |
toString() |
public static final CacheConfig DISABLED
public static final String CACHE_DATA_ON_READ_KEY
public static final String CACHE_BLOCKS_ON_WRITE_KEY
public static final String CACHE_INDEX_BLOCKS_ON_WRITE_KEY
public static final String CACHE_BLOOM_BLOCKS_ON_WRITE_KEY
public static final String CACHE_DATA_BLOCKS_COMPRESSED_KEY
public static final String EVICT_BLOCKS_ON_CLOSE_KEY
public static final String BUCKET_CACHE_PERSISTENT_PATH_KEY
file:/tmp/bucketcache.data
, then we will write the bucketcache data to the file
/tmp/bucketcache.data
but the metadata on where the data is in the supplied file
is an in-memory map that needs to be persisted across restarts. Where to store this
in-memory state is what you supply here: e.g. /tmp/bucketcache.map
.public static final String BUCKET_CACHE_WRITER_THREADS_KEY
public static final String BUCKET_CACHE_WRITER_QUEUE_KEY
public static final String BUCKET_CACHE_BUCKETS_KEY
public static final int DEFAULT_BUCKET_CACHE_WRITER_THREADS
public static final int DEFAULT_BUCKET_CACHE_WRITER_QUEUE
public static final String PREFETCH_BLOCKS_ON_OPEN_KEY
public static final String BLOCKCACHE_BLOCKSIZE_KEY
HConstants.DEFAULT_BLOCKSIZE
.
TODO: this config point is completely wrong, as it's used to determine the
target block size of BlockCache instances. Rename.public static final boolean DEFAULT_CACHE_DATA_ON_READ
public static final boolean DEFAULT_CACHE_DATA_ON_WRITE
public static final boolean DEFAULT_IN_MEMORY
public static final boolean DEFAULT_CACHE_INDEXES_ON_WRITE
public static final boolean DEFAULT_CACHE_BLOOMS_ON_WRITE
public static final boolean DEFAULT_EVICT_ON_CLOSE
public static final boolean DEFAULT_CACHE_DATA_COMPRESSED
public static final boolean DEFAULT_PREFETCH_ON_OPEN
public CacheConfig(org.apache.hadoop.conf.Configuration conf, ColumnFamilyDescriptor family)
conf
- hbase configurationfamily
- column family configurationpublic CacheConfig(org.apache.hadoop.conf.Configuration conf)
CacheConfig(Configuration, ColumnFamilyDescriptor)
conf
- hbase configurationCacheConfig(Configuration, ColumnFamilyDescriptor)
public CacheConfig(CacheConfig cacheConf)
cacheConf
- public boolean isBlockCacheEnabled()
public BlockCache getBlockCache()
public boolean shouldCacheDataOnRead()
public boolean shouldDropBehindCompaction()
public boolean shouldCacheBlockOnRead(BlockType.BlockCategory category)
public boolean isInMemory()
public boolean shouldCacheDataOnWrite()
public void setCacheDataOnWrite(boolean cacheDataOnWrite)
cacheDataOnWrite
- whether data blocks should be written to the cache
when an HFile is writtenpublic boolean shouldCacheIndexesOnWrite()
public boolean shouldCacheBloomsOnWrite()
public boolean shouldEvictOnClose()
public void setEvictOnClose(boolean evictOnClose)
evictOnClose
- whether blocks should be evicted from the cache when an
HFile reader is closedpublic boolean shouldCacheDataCompressed()
public boolean shouldCacheCompressed(BlockType.BlockCategory category)
BlockType.BlockCategory
should be compressed in blockcache, false otherwisepublic boolean shouldPrefetchOnOpen()
public boolean shouldReadBlockFromCache(BlockType blockType)
TODO: today family.isBlockCacheEnabled()
only means cacheDataOnRead
, so here we
consider lots of other configurations such as cacheDataOnWrite
. We should fix this in
the future, cacheDataOnWrite
should honor the CF level isBlockCacheEnabled
configuration.
public boolean shouldLockOnCacheMiss(BlockType blockType)
public static LruBlockCache getOnHeapCache(org.apache.hadoop.conf.Configuration c)
c
- Configuration to use.public CacheStats getOnHeapCacheStats()
public CacheStats getL2CacheStats()
public static BlockCache instantiateBlockCache(org.apache.hadoop.conf.Configuration conf)
null
in case none should be used.
Sets GLOBAL_BLOCK_CACHE_INSTANCEconf
- The current configuration.null
.Copyright © 2007–2019 Cloudera. All rights reserved.