org.apache.hadoop.hbase.master.assignment

Class RegionTransitionProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

Direct Known Subclasses:

AssignProcedure, UnassignProcedure

@InterfaceAudience.Private
public abstract class RegionTransitionProcedure
extends Procedure<MasterProcedureEnv>
implements TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

Base class for the Assign and Unassign Procedure. Locking: Takes exclusive lock on the region being assigned/unassigned. Thus, there can only be one RegionTransitionProcedure per region running at a time (see MasterProcedureScheduler).

This procedure is asynchronous and responds to external events. The AssignmentManager will notify this procedure when the RS completes the operation and reports the transitioned state (see the Assign and Unassign class for more detail).

Procedures move from the REGION_TRANSITION_QUEUE state when they are first submitted, to the REGION_TRANSITION_DISPATCH state when the request to remote server is sent and the Procedure is suspended waiting on external event to be woken again. Once the external event is triggered, Procedure moves to the REGION_TRANSITION_FINISH state.

NOTE: AssignProcedure and UnassignProcedure should not be thought of as being asymmetric, at least currently.

• AssignProcedure moves through all the above described states and implements methods associated with each while UnassignProcedure starts at state REGION_TRANSITION_DISPATCH and state REGION_TRANSITION_QUEUE is not supported.
• When any step in AssignProcedure fails, failure handler AssignProcedure#handleFailure(MasterProcedureEnv, RegionStateNode) re-attempts the assignment by setting the procedure state to REGION_TRANSITION_QUEUE and forces assignment to a different target server by setting AssignProcedure.forceNewPlan. When the number of attempts reaches threshold configuration 'hbase.assignment.maximum.attempts', the procedure is aborted. For UnassignProcedure, similar re-attempts are intentionally not implemented. It is a 'one shot' procedure. See its class doc for how it handles failure.
• If we find a region in an 'unexpected' state, we'll complain and retry with backoff forever. The 'unexpected' state needs to be fixed either by another running Procedure or by operator intervention (Regions in 'unexpected' state indicates bug or unexpected transition type). For this to work, subclasses need to persist the 'attempt' counter kept in this class when they do serializeStateData and restore it inside their deserializeStateData, just as they do for regionInfo.

TODO: Considering it is a priority doing all we can to get make a region available as soon as possible, re-attempting with any target makes sense if specified target fails in case of AssignProcedure. For UnassignProcedure, our concern is preventing data loss on failed unassign. See class doc for explanation.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicBoolean	aborted

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
RegionTransitionProcedure()
RegionTransitionProcedure(RegionInfo regionInfo, boolean override)

Method Summary

Modifier and Type	Method and Description
protected boolean	abort(MasterProcedureEnv env) The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable.
protected Procedure.LockState	acquireLock(MasterProcedureEnv env) The user should override this method if they need a lock on an Entity.
protected boolean	addToRemoteDispatcher(MasterProcedureEnv env, ServerName targetServer) Be careful! At the end of this method, the procedure has either succeeded and this procedure has been set into a suspended state OR, we failed and this procedure has been put back on the scheduler ready for another worker to pick it up.
protected void	bypass(MasterProcedureEnv env) Set the bypass to true.
protected Procedure[]	execute(MasterProcedureEnv env) The main code of the procedure.
protected abstract void	finishTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected int	getAttempt()
RegionInfo	getRegionInfo()
RegionStates.RegionStateNode	getRegionState(MasterProcedureEnv env)
abstract ServerName	getServer(MasterProcedureEnv env) Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.
TableName	getTableName()
protected MasterProcedureProtos.RegionTransitionState	getTransitionState()
protected boolean	holdLock(MasterProcedureEnv env) Used to keep the procedure lock even when the procedure is yielding or suspended.
boolean	isMeta()
protected abstract boolean	isRollbackSupported(MasterProcedureProtos.RegionTransitionState state)
protected boolean	isServerOnline(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
protected boolean	isServerOnline(MasterProcedureEnv env, ServerName serverName)
protected void	releaseLock(MasterProcedureEnv env) The user should override this method, and release lock if necessary.
abstract RemoteProcedureDispatcher.RemoteOperation	remoteCallBuild(MasterProcedureEnv env, ServerName serverName) For building the remote operation.
protected abstract boolean	remoteCallFailed(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, IOException exception)
boolean	remoteCallFailed(MasterProcedureEnv env, ServerName serverName, IOException exception) Called when the executeProcedure call is failed.
void	remoteOperationCompleted(MasterProcedureEnv env) Called when RS tells the remote procedure is succeeded through the reportProcedureDone method.
void	remoteOperationFailed(MasterProcedureEnv env, RemoteProcedureException error) Called when RS tells the remote procedure is failed through the reportProcedureDone method.
protected abstract void	reportTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode, RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long seqId)
protected void	reportTransition(MasterProcedureEnv env, ServerName serverName, RegionServerStatusProtos.RegionStateTransition.TransitionCode code, long seqId)
protected void	rollback(MasterProcedureEnv env) The code to undo what was done by the execute() code.
protected void	setAttempt(int attempt) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected void	setOverride(boolean override) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected void	setRegionInfo(RegionInfo regionInfo) This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.
protected boolean	setTimeoutFailure(MasterProcedureEnv env) At end of timeout, wake ourselves up so we run again.
protected boolean	shouldWaitClientAck(MasterProcedureEnv env) By default, the executor will keep the procedure result around util the eviction TTL is expired.
protected abstract boolean	startTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode)
boolean	storeInDispatchedQueue() Whether store this remote procedure in dispatched queue only OpenRegionProcedure and CloseRegionProcedure return false since they are not fully controlled by dispatcher
void	toStringClassDetails(StringBuilder sb) Extend the toString() information with the procedure details e.g.
protected void	toStringState(StringBuilder builder) Called from Procedure.toString() when interpolating Procedure State.
protected abstract boolean	updateTransition(MasterProcedureEnv env, RegionStates.RegionStateNode regionNode) Called when the Procedure is in the REGION_TRANSITION_DISPATCH state.
protected boolean	waitInitialized(MasterProcedureEnv env) The Procedure.doAcquireLock(Object, ProcedureStore) will be split into two steps, first, it will call us to determine whether we need to wait for initialization, second, it will call Procedure.acquireLock(Object) to actually handle the lock for this procedure.

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, compareTo, completionCleanup, deserializeStateData, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcedureMetrics, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, isYieldAfterExecutionStep, removeStackIndex, serializeStateData, setAbortFailure, setChildrenLatch, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, skipPersistence, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

getTableOperationType

Field Detail

aborted

protected final AtomicBoolean aborted

Constructor Detail

RegionTransitionProcedure

public RegionTransitionProcedure()

RegionTransitionProcedure

public RegionTransitionProcedure(RegionInfo regionInfo,
                                 boolean override)

Method Detail

getRegionInfo

public RegionInfo getRegionInfo()

setRegionInfo

protected void setRegionInfo(RegionInfo regionInfo)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method. Expectation is that subclasses will persist `regioninfo` in their Procedure.serializeStateData(ProcedureStateSerializer) method and then restore `regionInfo` on deserialization by calling this.

setOverride

protected void setOverride(boolean override)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method. Expectation is that subclasses will persist `override` in their Procedure.serializeStateData(ProcedureStateSerializer) method and then restore `override` on deserialization by calling this.

setAttempt

protected void setAttempt(int attempt)

This setter is for subclasses to call in their Procedure.deserializeStateData(ProcedureStateSerializer) method.

See Also:

setRegionInfo(RegionInfo)

getAttempt

protected int getAttempt()

getTableName

public TableName getTableName()

Specified by:

getTableName in interface TableProcedureInterface

Returns:

the name of the table the procedure is operating on

isMeta

public boolean isMeta()

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Description copied from class: Procedure

Extend the toString() information with the procedure details e.g. className and parameters

Overrides:

toStringClassDetails in class Procedure<MasterProcedureEnv>

Parameters:

sb - the string builder to use to append the proc specific information

getRegionState

public RegionStates.RegionStateNode getRegionState(MasterProcedureEnv env)

getTransitionState

protected MasterProcedureProtos.RegionTransitionState getTransitionState()

startTransition

protected abstract boolean startTransition(MasterProcedureEnv env,
                                           RegionStates.RegionStateNode regionNode)
                                    throws IOException,
                                           ProcedureSuspendedException

Throws:

IOException

ProcedureSuspendedException

updateTransition

protected abstract boolean updateTransition(MasterProcedureEnv env,
                                            RegionStates.RegionStateNode regionNode)
                                     throws IOException,
                                            ProcedureSuspendedException

Called when the Procedure is in the REGION_TRANSITION_DISPATCH state. In here we do the RPC call to OPEN/CLOSE the region. The suspending of the thread so it sleeps until it gets update that the OPEN/CLOSE has succeeded is complicated. Read the implementations to learn more.

Throws:

IOException

ProcedureSuspendedException

finishTransition

protected abstract void finishTransition(MasterProcedureEnv env,
                                         RegionStates.RegionStateNode regionNode)
                                  throws IOException,
                                         ProcedureSuspendedException

Throws:

IOException

ProcedureSuspendedException

reportTransition

protected abstract void reportTransition(MasterProcedureEnv env,
                                         RegionStates.RegionStateNode regionNode,
                                         RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                         long seqId)
                                  throws UnexpectedStateException

Throws:

UnexpectedStateException

remoteCallBuild

public abstract RemoteProcedureDispatcher.RemoteOperation remoteCallBuild(MasterProcedureEnv env,
                                                                          ServerName serverName)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

For building the remote operation.

Specified by:

remoteCallBuild in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

remoteCallFailed

protected abstract boolean remoteCallFailed(MasterProcedureEnv env,
                                            RegionStates.RegionStateNode regionNode,
                                            IOException exception)

Returns:

True if processing of fail is complete; the procedure will be woken from its suspend and we'll go back to running through procedure steps: otherwise if false we leave the procedure in suspended state.

remoteCallFailed

public boolean remoteCallFailed(MasterProcedureEnv env,
                                ServerName serverName,
                                IOException exception)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

Called when the executeProcedure call is failed.

Specified by:

remoteCallFailed in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

addToRemoteDispatcher

protected boolean addToRemoteDispatcher(MasterProcedureEnv env,
                                        ServerName targetServer)

Be careful! At the end of this method, the procedure has either succeeded and this procedure has been set into a suspended state OR, we failed and this procedure has been put back on the scheduler ready for another worker to pick it up. In both cases, we need to exit the current Worker processing immediately!

Returns:

True if we successfully dispatched the call and false if we failed; if failed, we need to roll back any setup done for the dispatch.

storeInDispatchedQueue

public boolean storeInDispatchedQueue()

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

Whether store this remote procedure in dispatched queue only OpenRegionProcedure and CloseRegionProcedure return false since they are not fully controlled by dispatcher

Specified by:

storeInDispatchedQueue in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

reportTransition

protected void reportTransition(MasterProcedureEnv env,
                                ServerName serverName,
                                RegionServerStatusProtos.RegionStateTransition.TransitionCode code,
                                long seqId)
                         throws UnexpectedStateException

Throws:

UnexpectedStateException

isServerOnline

protected boolean isServerOnline(MasterProcedureEnv env,
                                 RegionStates.RegionStateNode regionNode)

isServerOnline

protected boolean isServerOnline(MasterProcedureEnv env,
                                 ServerName serverName)

toStringState

protected void toStringState(StringBuilder builder)

Description copied from class: Procedure

Called from Procedure.toString() when interpolating Procedure State. Allows decorating generic Procedure State with Procedure particulars.

Overrides:

toStringState in class Procedure<MasterProcedureEnv>

Parameters:

builder - Append current ProcedureProtos.ProcedureState

execute

protected Procedure[] execute(MasterProcedureEnv env)
                       throws ProcedureSuspendedException

Description copied from class: Procedure

The main code of the procedure. It must be idempotent since execute() may be called multiple times in case of machine failure in the middle of the execution.

Specified by:

execute in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

a set of sub-procedures to run or ourselves if there is more work to do or null if the procedure is done.

Throws:

ProcedureSuspendedException - Signal to the executor that Procedure has suspended itself and has set itself up waiting for an external event to wake it back up again.

setTimeoutFailure

protected boolean setTimeoutFailure(MasterProcedureEnv env)

At end of timeout, wake ourselves up so we run again.

Overrides:

setTimeoutFailure in class Procedure<MasterProcedureEnv>

Returns:

true to let the framework handle the timeout as abort, false in case the procedure handled the timeout itself.

rollback

protected void rollback(MasterProcedureEnv env)

Description copied from class: Procedure

The code to undo what was done by the execute() code. It is called when the procedure or one of the sub-procedures failed or an abort was requested. It should cleanup all the resources created by the execute() call. The implementation must be idempotent since rollback() may be called multiple time in case of machine failure in the middle of the execution.

Specified by:

rollback in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

isRollbackSupported

protected abstract boolean isRollbackSupported(MasterProcedureProtos.RegionTransitionState state)

abort

protected boolean abort(MasterProcedureEnv env)

Description copied from class: Procedure

The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable. The simplest implementation is to have an AtomicBoolean set in the abort() method and then the execute() will check if the abort flag is set or not. abort() may be called multiple times from the client, so the implementation must be idempotent.

NOTE: abort() is not like Thread.interrupt(). It is just a notification that allows the procedure implementor abort.

Specified by:

abort in class Procedure<MasterProcedureEnv>

waitInitialized

protected boolean waitInitialized(MasterProcedureEnv env)

Description copied from class: Procedure

The Procedure.doAcquireLock(Object, ProcedureStore) will be split into two steps, first, it will call us to determine whether we need to wait for initialization, second, it will call Procedure.acquireLock(Object) to actually handle the lock for this procedure.

This is because that when master restarts, we need to restore the lock state for all the procedures to not break the semantic if Procedure.holdLock(Object) is true. But the ProcedureExecutor will be started before the master finish initialization(as it is part of the initialization!), so we need to split the code into two steps, and when restore, we just restore the lock part and ignore the waitInitialized part. Otherwise there will be dead lock.

Overrides:

waitInitialized in class Procedure<MasterProcedureEnv>

Returns:

true means we need to wait until the environment has been initialized, otherwise true.

acquireLock

protected Procedure.LockState acquireLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method if they need a lock on an Entity. A lock can be anything, and it is up to the implementor. The Procedure Framework will call this method just before it invokes Procedure.execute(Object). It calls Procedure.releaseLock(Object) after the call to execute.

If you need to hold the lock for the life of the Procedure -- i.e. you do not want any other Procedure interfering while this Procedure is running, see Procedure.holdLock(Object).

Example: in our Master we can execute request in parallel for different tables. We can create t1 and create t2 and these creates can be executed at the same time. Anything else on t1/t2 is queued waiting that specific table create to happen.

There are 3 LockState:

• LOCK_ACQUIRED should be returned when the proc has the lock and the proc is ready to execute.
• LOCK_YIELD_WAIT should be returned when the proc has not the lock and the framework should take care of readding the procedure back to the runnable set for retry
• LOCK_EVENT_WAIT should be returned when the proc has not the lock and someone will take care of readding the procedure back to the runnable set when the lock is available.

Overrides:

acquireLock in class Procedure<MasterProcedureEnv>

Returns:

the lock state as described above.

releaseLock

protected void releaseLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method, and release lock if necessary.

Overrides:

releaseLock in class Procedure<MasterProcedureEnv>

holdLock

protected boolean holdLock(MasterProcedureEnv env)

Description copied from class: Procedure

Used to keep the procedure lock even when the procedure is yielding or suspended.

Overrides:

holdLock in class Procedure<MasterProcedureEnv>

Returns:

true if the procedure should hold on the lock until completionCleanup()

shouldWaitClientAck

protected boolean shouldWaitClientAck(MasterProcedureEnv env)

Description copied from class: Procedure

By default, the executor will keep the procedure result around util the eviction TTL is expired. The client can cut down the waiting time by requesting that the result is removed from the executor. In case of system started procedure, we can force the executor to auto-ack.

Overrides:

shouldWaitClientAck in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

true if the executor should wait the client ack for the result. Defaults to return true.

getServer

public abstract ServerName getServer(MasterProcedureEnv env)

Used by ServerCrashProcedure to see if this Assign/Unassign needs processing.

Returns:

ServerName the Assign or Unassign is going against.

remoteOperationCompleted

public void remoteOperationCompleted(MasterProcedureEnv env)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

Called when RS tells the remote procedure is succeeded through the reportProcedureDone method.

Specified by:

remoteOperationCompleted in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

remoteOperationFailed

public void remoteOperationFailed(MasterProcedureEnv env,
                                  RemoteProcedureException error)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

Called when RS tells the remote procedure is failed through the reportProcedureDone method.

Specified by:

remoteOperationFailed in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

bypass

protected void bypass(MasterProcedureEnv env)

Description copied from class: Procedure

Set the bypass to true. Only called in ProcedureExecutor#bypassProcedure(long, long, boolean) for now. DO NOT use this method alone, since we can't just bypass one single procedure. We need to bypass its ancestor too. If your Procedure has set state, it needs to undo it in here.

Overrides:

bypass in class Procedure<MasterProcedureEnv>

Parameters:

env - Current environment. May be null because of context; e.g. pretty-printing procedure WALs where there is no 'environment' (and where Procedures that require an 'environment' won't be run.