org.apache.hadoop.hbase.master.procedure

Class MasterProcedureScheduler

java.lang.Object

org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler

org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler

All Implemented Interfaces:

ProcedureScheduler

@InterfaceAudience.Private
public class MasterProcedureScheduler
extends AbstractProcedureScheduler

ProcedureScheduler for the Master Procedures. This ProcedureScheduler tries to provide to the ProcedureExecutor procedures that can be executed without having to wait on a lock. Most of the master operations can be executed concurrently, if they are operating on different tables (e.g. two create table procedures can be performed at the same time) or against two different servers; say two servers that crashed at about the same time.

Each procedure should implement an Interface providing information for this queue. For example table related procedures should implement TableProcedureInterface. Each procedure will be pushed in its own queue, and based on the operation type we may make smarter decisions: e.g. we can abort all the operations preceding a delete table, or similar.

Concurrency control

Concurrent access to member variables (tableRunQueue, serverRunQueue, locking, tableMap, serverBuckets) is controlled by schedLock(). This mainly includes:

• AbstractProcedureScheduler.push(Procedure, boolean, boolean): A push will add a Queue back to run-queue when:
  1. Queue was empty before push (so must have been out of run-queue)
  2. Child procedure is added (which means parent procedure holds exclusive lock, and it must have moved Queue out of run-queue)
• AbstractProcedureScheduler.poll(long): A poll will remove a Queue from run-queue when:
  1. Queue becomes empty after poll
  2. Exclusive lock is requested by polled procedure and lock is available (returns the procedure)
  3. Exclusive lock is requested but lock is not available (returns null)
  4. Polled procedure is child of parent holding exclusive lock and the next procedure is not a child
• Namespace/table/region locks: Queue is added back to run-queue when lock being released is:
  1. Exclusive lock
  2. Last shared lock (in case queue was removed because next procedure in queue required exclusive lock)

Constructor Summary

Constructors

Constructor and Description
MasterProcedureScheduler(Function<Long,Procedure<?>> procedureRetriever)

Method Summary

Modifier and Type	Method and Description
void	clear() Clear current state of scheduler such that it is equivalent to newly created scheduler.
void	completionCleanup(Procedure proc) The procedure in execution completed.
protected Procedure	dequeue(boolean onlyUrgent)
String	dumpLocks() For debugging.
protected void	enqueue(Procedure proc, boolean addFront) Add the procedure to the queue.
LockedResource	getLockResource(LockedResourceType resourceType, String resourceName)
List<LockedResource>	getLocks() List lock queues.
protected boolean	queueHasRunnables() Returns true if there are procedures available to process.
protected int	queueSize() Returns the number of elements in this queue.
boolean	waitMetaExclusiveLock(Procedure<?> procedure) Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.
boolean	waitNamespaceExclusiveLock(Procedure<?> procedure, String namespace) Suspend the procedure if the specified namespace is already locked.
boolean	waitPeerExclusiveLock(Procedure<?> procedure, String peerId) Try to acquire the exclusive lock on the specified peer.
boolean	waitRegion(Procedure<?> procedure, RegionInfo regionInfo) Suspend the procedure if the specified region is already locked.
boolean	waitRegions(Procedure<?> procedure, TableName table, RegionInfo... regionInfo) Suspend the procedure if the specified set of regions are already locked.
boolean	waitServerExclusiveLock(Procedure<?> procedure, ServerName serverName) Try to acquire the exclusive lock on the specified server.
boolean	waitTableExclusiveLock(Procedure<?> procedure, TableName table) Suspend the procedure if the specified table is already locked.
boolean	waitTableSharedLock(Procedure<?> procedure, TableName table) Suspend the procedure if the specified table is already locked.
void	wakeMetaExclusiveLock(Procedure<?> procedure) Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.
void	wakeNamespaceExclusiveLock(Procedure<?> procedure, String namespace) Wake the procedures waiting for the specified namespace
void	wakePeerExclusiveLock(Procedure<?> procedure, String peerId) Wake the procedures waiting for the specified peer
void	wakeRegion(Procedure<?> procedure, RegionInfo regionInfo) Wake the procedures waiting for the specified region
void	wakeRegions(Procedure<?> procedure, TableName table, RegionInfo... regionInfo) Wake the procedures waiting for the specified regions
void	wakeServerExclusiveLock(Procedure<?> procedure, ServerName serverName) Wake the procedures waiting for the specified server
void	wakeTableExclusiveLock(Procedure<?> procedure, TableName table) Wake the procedures waiting for the specified table
void	wakeTableSharedLock(Procedure<?> procedure, TableName table) Wake the procedures waiting for the specified table
void	yield(Procedure proc) The procedure can't run at the moment.

Methods inherited from class org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler

addBack, addBack, addFront, addFront, addFront, dequeue, getNullPollCalls, getPollCalls, hasRunnables, poll, poll, poll, poll, poll, poll, push, schedLock, schedUnlock, signalAll, size, start, stop, waitProcedure, wakePollIfNeeded, wakeProcedure, wakeWaitingProcedures

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MasterProcedureScheduler

public MasterProcedureScheduler(Function<Long,Procedure<?>> procedureRetriever)

Method Detail

yield

public void yield(Procedure proc)

Description copied from interface: ProcedureScheduler

The procedure can't run at the moment. add it back to the queue, giving priority to someone else.

Parameters:

proc - the Procedure to add back to the list

enqueue

protected void enqueue(Procedure proc,
                       boolean addFront)

Description copied from class: AbstractProcedureScheduler

Add the procedure to the queue. NOTE: this method is called with the sched lock held.

Specified by:

enqueue in class AbstractProcedureScheduler

Parameters:

proc - the Procedure to add

addFront - true if the item should be added to the front of the queue

queueHasRunnables

protected boolean queueHasRunnables()

Description copied from class: AbstractProcedureScheduler

Returns true if there are procedures available to process. NOTE: this method is called with the sched lock held.

Specified by:

queueHasRunnables in class AbstractProcedureScheduler

Returns:

true if there are procedures available to process, otherwise false.

dequeue

protected Procedure dequeue(boolean onlyUrgent)

Specified by:

dequeue in class AbstractProcedureScheduler

getLocks

public List<LockedResource> getLocks()

Description copied from interface: ProcedureScheduler

List lock queues.

Returns:

the locks

getLockResource

public LockedResource getLockResource(LockedResourceType resourceType,
                                      String resourceName)

Returns:

LockedResource for resource of specified type & name. null if resource is not locked.

clear

public void clear()

Description copied from interface: ProcedureScheduler

Clear current state of scheduler such that it is equivalent to newly created scheduler. Used for testing failure and recovery. To emulate server crash/restart, ProcedureExecutor resets its own state and calls clear() on scheduler.

queueSize

protected int queueSize()

Description copied from class: AbstractProcedureScheduler

Returns the number of elements in this queue. NOTE: this method is called with the sched lock held.

Specified by:

queueSize in class AbstractProcedureScheduler

Returns:

the number of elements in this queue.

completionCleanup

public void completionCleanup(Procedure proc)

Description copied from interface: ProcedureScheduler

The procedure in execution completed. This can be implemented to perform cleanups.

Parameters:

proc - the Procedure that completed the execution.

waitTableExclusiveLock

public boolean waitTableExclusiveLock(Procedure<?> procedure,
                                      TableName table)

Suspend the procedure if the specified table is already locked. Other operations in the table-queue will be executed after the lock is released.

Parameters:

procedure - the procedure trying to acquire the lock

table - Table to lock

Returns:

true if the procedure has to wait for the table to be available

wakeTableExclusiveLock

public void wakeTableExclusiveLock(Procedure<?> procedure,
                                   TableName table)

Wake the procedures waiting for the specified table

Parameters:

procedure - the procedure releasing the lock

table - the name of the table that has the exclusive lock

waitTableSharedLock

public boolean waitTableSharedLock(Procedure<?> procedure,
                                   TableName table)

Suspend the procedure if the specified table is already locked. other "read" operations in the table-queue may be executed concurrently,

Parameters:

procedure - the procedure trying to acquire the lock

table - Table to lock

Returns:

true if the procedure has to wait for the table to be available

wakeTableSharedLock

public void wakeTableSharedLock(Procedure<?> procedure,
                                TableName table)

Wake the procedures waiting for the specified table

Parameters:

procedure - the procedure releasing the lock

table - the name of the table that has the shared lock

waitRegion

public boolean waitRegion(Procedure<?> procedure,
                          RegionInfo regionInfo)

Suspend the procedure if the specified region is already locked.

Parameters:

procedure - the procedure trying to acquire the lock on the region

regionInfo - the region we are trying to lock

Returns:

true if the procedure has to wait for the regions to be available

waitRegions

public boolean waitRegions(Procedure<?> procedure,
                           TableName table,
                           RegionInfo... regionInfo)

Suspend the procedure if the specified set of regions are already locked.

Parameters:

procedure - the procedure trying to acquire the lock on the regions

table - the table name of the regions we are trying to lock

regionInfo - the list of regions we are trying to lock

Returns:

true if the procedure has to wait for the regions to be available

wakeRegion

public void wakeRegion(Procedure<?> procedure,
                       RegionInfo regionInfo)

Wake the procedures waiting for the specified region

Parameters:

procedure - the procedure that was holding the region

regionInfo - the region the procedure was holding

wakeRegions

public void wakeRegions(Procedure<?> procedure,
                        TableName table,
                        RegionInfo... regionInfo)

Wake the procedures waiting for the specified regions

Parameters:

procedure - the procedure that was holding the regions

regionInfo - the list of regions the procedure was holding

waitNamespaceExclusiveLock

public boolean waitNamespaceExclusiveLock(Procedure<?> procedure,
                                          String namespace)

Suspend the procedure if the specified namespace is already locked.

Parameters:

procedure - the procedure trying to acquire the lock

namespace - Namespace to lock

Returns:

true if the procedure has to wait for the namespace to be available

See Also:

wakeNamespaceExclusiveLock(Procedure,String)

wakeNamespaceExclusiveLock

public void wakeNamespaceExclusiveLock(Procedure<?> procedure,
                                       String namespace)

Wake the procedures waiting for the specified namespace

Parameters:

procedure - the procedure releasing the lock

namespace - the namespace that has the exclusive lock

See Also:

waitNamespaceExclusiveLock(Procedure,String)

waitServerExclusiveLock

public boolean waitServerExclusiveLock(Procedure<?> procedure,
                                       ServerName serverName)

Try to acquire the exclusive lock on the specified server.

Parameters:

procedure - the procedure trying to acquire the lock

serverName - Server to lock

Returns:

true if the procedure has to wait for the server to be available

See Also:

wakeServerExclusiveLock(Procedure,ServerName)

wakeServerExclusiveLock

public void wakeServerExclusiveLock(Procedure<?> procedure,
                                    ServerName serverName)

Wake the procedures waiting for the specified server

Parameters:

procedure - the procedure releasing the lock

serverName - the server that has the exclusive lock

See Also:

waitServerExclusiveLock(Procedure,ServerName)

waitPeerExclusiveLock

public boolean waitPeerExclusiveLock(Procedure<?> procedure,
                                     String peerId)

Try to acquire the exclusive lock on the specified peer.

Parameters:

procedure - the procedure trying to acquire the lock

peerId - peer to lock

Returns:

true if the procedure has to wait for the peer to be available

See Also:

wakePeerExclusiveLock(Procedure, String)

wakePeerExclusiveLock

public void wakePeerExclusiveLock(Procedure<?> procedure,
                                  String peerId)

Wake the procedures waiting for the specified peer

Parameters:

procedure - the procedure releasing the lock

peerId - the peer that has the exclusive lock

See Also:

waitPeerExclusiveLock(Procedure, String)

waitMetaExclusiveLock

@Deprecated
public boolean waitMetaExclusiveLock(Procedure<?> procedure)

Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.

Try to acquire the exclusive lock on meta.

Parameters:

procedure - the procedure trying to acquire the lock

Returns:

true if the procedure has to wait for meta to be available

See Also:

wakeMetaExclusiveLock(Procedure)

wakeMetaExclusiveLock

@Deprecated
public void wakeMetaExclusiveLock(Procedure<?> procedure)

Deprecated. only used for RecoverMetaProcedure. Should be removed along with RecoverMetaProcedure.

Wake the procedures waiting for meta.

Parameters:

procedure - the procedure releasing the lock

See Also:

waitMetaExclusiveLock(Procedure)

dumpLocks

public String dumpLocks()
                 throws IOException

For debugging. Expensive.

Throws:

IOException