CompactingMemStore (Apache HBase 2.1.0-cdh6.3.2 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.AbstractMemStore
- - org.apache.hadoop.hbase.regionserver.CompactingMemStore

All Implemented Interfaces:

MemStore
```
@InterfaceAudience.Private
public class CompactingMemStore
extends AbstractMemStore
```
A memstore implementation which supports in-memory compaction. A compaction pipeline is added between the active set and the snapshot data structures; it consists of a list of segments that are subject to compaction. Like the snapshot, all pipeline segments are read-only; updates only affect the active set. To ensure this property we take advantage of the existing blocking mechanism -- the active set is pushed to the pipeline while holding the region's updatesLock in exclusive mode. Periodically, a compaction is applied in the background to all pipeline segments resulting in a single read-only component. The ``old'' segments are discarded when no scanner is reading them.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`CompactingMemStore.IndexType` Types of indexes (part of immutable segments) to be used after flattening, compaction, or merge are applied.

Field Summary

Fields
Modifier and Type	Field and Description
`protected AtomicBoolean`	`allowCompaction`
`static String`	`COMPACTING_MEMSTORE_TYPE_DEFAULT`
`static String`	`COMPACTING_MEMSTORE_TYPE_KEY`
`protected MemStoreCompactor`	`compactor`
`static long`	`DEEP_OVERHEAD`
`static int`	`IN_MEMORY_CONPACTION_POOL_SIZE_DEFAULT`
`static String`	`IN_MEMORY_CONPACTION_POOL_SIZE_KEY`
`static String`	`IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY`

Fields inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore
active, FIXED_OVERHEAD, regionServices, snapshot, snapshotId

Constructor Summary

Constructors
Constructor and Description
`CompactingMemStore(org.apache.hadoop.conf.Configuration conf, CellComparator c, HStore store, RegionServicesForStores regionServices, MemoryCompactionPolicy compactionPolicy)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected void`	`checkActiveSize()` Check whether anything need to be done based on the current active set size.
`protected List<KeyValueScanner>`	`createList(int capacity)`
`protected MemStoreCompactor`	`createMemStoreCompactor(MemoryCompactionPolicy compactionPolicy)`
`void`	`debug()`
`void`	`flattenOneSegment(long requesterVersion, MemStoreCompactionStrategy.Action action)`
`String`	`getFamilyName()`
`MemStoreSize`	`getFlushableSize()` Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set).
`VersionedSegmentsList`	`getImmutableSegments()`
`CompactingMemStore.IndexType`	`getIndexType()`
`List<KeyValueScanner>`	`getScanners(long readPt)`
`protected List<Segment>`	`getSegments()`
`long`	`getSmallestReadPoint()`
`HStore`	`getStore()`
`boolean`	`hasImmutableSegments()`
`protected long`	`heapSize()`
`boolean`	`isSloppy()`
`protected long`	`keySize()`
`long`	`preFlushSeqIDEstimation()` This method is called before the flush is executed.
`protected void`	`pushActiveToPipeline(MutableSegment active)`
`void`	`setCompositeSnapshot(boolean useCompositeSnapshot)`
`protected boolean`	`shouldFlushInMemory()`
`MemStoreSize`	`size()`
`MemStoreSnapshot`	`snapshot()` Push the current active memstore segment into the pipeline and create a snapshot of the tail of current compaction pipeline Snapshot must be cleared by call to `AbstractMemStore.clearSnapshot(long)`.
`void`	`startReplayingFromWAL()` This message intends to inform the MemStore that next coming updates are going to be part of the replaying edits from WAL
`void`	`stopReplayingFromWAL()` This message intends to inform the MemStore that the replaying edits from WAL are done
`boolean`	`swapCompactedSegments(VersionedSegmentsList versionedList, ImmutableSegment result, boolean merge)`
`void`	`updateLowestUnflushedSequenceIdInWAL(boolean onlyIfGreater)` Updates the wal with the lowest sequence id (oldest entry) that is still in memory

Methods inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore
add, add, addToScanners, addToScanners, clearSnapshot, dump, getComparator, getConfiguration, getLowest, getNextRow, getSnapshotSize, resetActive, timeOfOldestEdit, toString, upsert

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - COMPACTING_MEMSTORE_TYPE_KEY
```
public static final String COMPACTING_MEMSTORE_TYPE_KEY
```
    See Also:
    
    Constant Field Values
  - COMPACTING_MEMSTORE_TYPE_DEFAULT
```
public static final String COMPACTING_MEMSTORE_TYPE_DEFAULT
```
  - IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY
```
public static final String IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY
```
    See Also:
    
    Constant Field Values
  - IN_MEMORY_CONPACTION_POOL_SIZE_KEY
```
public static final String IN_MEMORY_CONPACTION_POOL_SIZE_KEY
```
    See Also:
    
    Constant Field Values
  - IN_MEMORY_CONPACTION_POOL_SIZE_DEFAULT
```
public static final int IN_MEMORY_CONPACTION_POOL_SIZE_DEFAULT
```
    See Also:
    
    Constant Field Values
  - compactor
```
protected MemStoreCompactor compactor
```
  - allowCompaction
```
protected final AtomicBoolean allowCompaction
```
  - DEEP_OVERHEAD
```
public static final long DEEP_OVERHEAD
```
- Constructor Detail
  - CompactingMemStore
```
public CompactingMemStore(org.apache.hadoop.conf.Configuration conf,
                          CellComparator c,
                          HStore store,
                          RegionServicesForStores regionServices,
                          MemoryCompactionPolicy compactionPolicy)
                   throws IOException
```
    Throws:
    
    IOException
- Method Detail
  - createMemStoreCompactor
```
protected MemStoreCompactor createMemStoreCompactor(MemoryCompactionPolicy compactionPolicy)
                                             throws IllegalArgumentIOException
```
    Throws:
    
    IllegalArgumentIOException
  - size
```
public MemStoreSize size()
```
    Returns:
    
    Total memory occupied by this MemStore. This won't include any size occupied by the snapshot. We assume the snapshot will get cleared soon. This is not thread safe and the memstore may be changed while computing its size. It is the responsibility of the caller to make sure this doesn't happen.
  - preFlushSeqIDEstimation
```
public long preFlushSeqIDEstimation()
```
    This method is called before the flush is executed.
    
    Returns:
    
    an estimation (lower bound) of the unflushed sequence id in memstore after the flush is executed. if memstore will be cleared returns HConstants.NO_SEQNUM.
  - isSloppy
```
public boolean isSloppy()
```
  - snapshot
```
public MemStoreSnapshot snapshot()
```
    Push the current active memstore segment into the pipeline and create a snapshot of the tail of current compaction pipeline Snapshot must be cleared by call to AbstractMemStore.clearSnapshot(long). AbstractMemStore.clearSnapshot(long).
    
    Returns:
    
    MemStoreSnapshot
  - getFlushableSize
```
public MemStoreSize getFlushableSize()
```
    Description copied from interface: MemStore
    
    Flush will first clear out the data in snapshot if any (It will take a second flush invocation to clear the current Cell set). If snapshot is empty, current Cell set will be flushed.
    
    Returns:
    
    On flush, how much memory we will clear.
  - keySize
```
protected long keySize()
```
    Specified by:
    
    keySize in class AbstractMemStore
    
    Returns:
    
    The total size of cells in this memstore. We will not consider cells in the snapshot
  - heapSize
```
protected long heapSize()
```
    Specified by:
    
    heapSize in class AbstractMemStore
    
    Returns:
    
    The total heap size of cells in this memstore. We will not consider cells in the snapshot
  - updateLowestUnflushedSequenceIdInWAL
```
public void updateLowestUnflushedSequenceIdInWAL(boolean onlyIfGreater)
```
    Description copied from class: AbstractMemStore
    
    Updates the wal with the lowest sequence id (oldest entry) that is still in memory
    
    Specified by:
    
    updateLowestUnflushedSequenceIdInWAL in class AbstractMemStore
    
    Parameters:
    
    onlyIfGreater - a flag that marks whether to update the sequence id no matter what or only if it is greater than the previous sequence id
  - startReplayingFromWAL
```
public void startReplayingFromWAL()
```
    This message intends to inform the MemStore that next coming updates are going to be part of the replaying edits from WAL
  - stopReplayingFromWAL
```
public void stopReplayingFromWAL()
```
    This message intends to inform the MemStore that the replaying edits from WAL are done
  - getSegments
```
protected List<Segment> getSegments()
```
    Specified by:
    
    getSegments in class AbstractMemStore
    
    Returns:
    
    an ordered list of segments from most recent to oldest in memstore
  - setCompositeSnapshot
```
public void setCompositeSnapshot(boolean useCompositeSnapshot)
```
  - swapCompactedSegments
```
public boolean swapCompactedSegments(VersionedSegmentsList versionedList,
                                     ImmutableSegment result,
                                     boolean merge)
```
  - flattenOneSegment
```
public void flattenOneSegment(long requesterVersion,
                              MemStoreCompactionStrategy.Action action)
```
    Parameters:
    
    requesterVersion - The caller must hold the VersionedList of the pipeline with version taken earlier. This version must be passed as a parameter here. The flattening happens only if versions match.
  - getIndexType
```
public CompactingMemStore.IndexType getIndexType()
```
  - hasImmutableSegments
```
public boolean hasImmutableSegments()
```
  - getImmutableSegments
```
public VersionedSegmentsList getImmutableSegments()
```
  - getSmallestReadPoint
```
public long getSmallestReadPoint()
```
  - getStore
```
public HStore getStore()
```
  - getFamilyName
```
public String getFamilyName()
```
  - getScanners
```
public List<KeyValueScanner> getScanners(long readPt)
                                  throws IOException
```
    Returns:
    
    scanner over the memstore. This might include scanner over the snapshot when one is present.
    
    Throws:
    
    IOException
  - createList
```
protected List<KeyValueScanner> createList(int capacity)
```
  - checkActiveSize
```
protected void checkActiveSize()
```
    Check whether anything need to be done based on the current active set size. The method is invoked upon every addition to the active set. For CompactingMemStore, flush the active set to the read-only memory if it's size is above threshold
    
    Specified by:
    
    checkActiveSize in class AbstractMemStore
  - shouldFlushInMemory
```
protected boolean shouldFlushInMemory()
```
  - pushActiveToPipeline
```
protected void pushActiveToPipeline(MutableSegment active)
```
  - debug
```
public void debug()
```

Class CompactingMemStore

Nested Class Summary

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.regionserver.AbstractMemStore

Methods inherited from class java.lang.Object

Field Detail

COMPACTING_MEMSTORE_TYPE_KEY

COMPACTING_MEMSTORE_TYPE_DEFAULT

IN_MEMORY_FLUSH_THRESHOLD_FACTOR_KEY

IN_MEMORY_CONPACTION_POOL_SIZE_KEY

IN_MEMORY_CONPACTION_POOL_SIZE_DEFAULT

compactor

allowCompaction

DEEP_OVERHEAD

Constructor Detail

CompactingMemStore

Method Detail

createMemStoreCompactor

size

preFlushSeqIDEstimation

isSloppy

snapshot

getFlushableSize

keySize

heapSize

updateLowestUnflushedSequenceIdInWAL

startReplayingFromWAL

stopReplayingFromWAL

getSegments

setCompositeSnapshot

swapCompactedSegments

flattenOneSegment

getIndexType

hasImmutableSegments

getImmutableSegments

getSmallestReadPoint

getStore

getFamilyName

getScanners

createList

checkActiveSize

shouldFlushInMemory

pushActiveToPipeline

debug