org.apache.hadoop.hbase.procedure2

Class ProcedureExecutor<TEnvironment>

java.lang.Object

org.apache.hadoop.hbase.procedure2.ProcedureExecutor<TEnvironment>

@InterfaceAudience.Private
public class ProcedureExecutor<TEnvironment>
extends Object

Thread Pool that executes the submitted procedures. The executor has a ProcedureStore associated. Each operation is logged and on restart the pending procedures are resumed. Unless the Procedure code throws an error (e.g. invalid user input) the procedure will complete (at some point in time), On restart the pending procedures are resumed and the once failed will be rolledback. The user can add procedures to the executor via submitProcedure(proc) check for the finished state via isFinished(procId) and get the result via getResult(procId)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	ProcedureExecutor.ProcedureExecutorListener
static class	ProcedureExecutor.Testing Class with parameters describing how to fail/die when in testing-context.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CHECK_OWNER_SET_CONF_KEY
static String	EVICT_ACKED_TTL_CONF_KEY
static String	EVICT_TTL_CONF_KEY
static String	UPGRADE_TO_2_2
static String	WORKER_KEEP_ALIVE_TIME_CONF_KEY

Constructor Summary

Constructors

Constructor and Description
ProcedureExecutor(org.apache.hadoop.conf.Configuration conf, TEnvironment environment, ProcedureStore store)
ProcedureExecutor(org.apache.hadoop.conf.Configuration conf, TEnvironment environment, ProcedureStore store, ProcedureScheduler scheduler)

Method Summary

Modifier and Type	Method and Description
boolean	abort(long procId) Send an abort notification the specified procedure.
boolean	abort(long procId, boolean mayInterruptIfRunning) Send an abort notification to the specified procedure.
void	addChore(ProcedureInMemoryChore<TEnvironment> chore) Add a chore procedure to the executor
List<Boolean>	bypassProcedure(List<Long> pids, long lockWait, boolean force, boolean recursive) Bypass a procedure.
NonceKey	createNonceKey(long nonceGroup, long nonce) Create a NoneKey from the specified nonceGroup and nonce.
int	getActiveExecutorCount()
Collection<Procedure<TEnvironment>>	getActiveProceduresNoCopy() Should only be used when starting up, where the procedure workers have not been started.
Set<Long>	getActiveProcIds()
int	getCorePoolSize()
TEnvironment	getEnvironment()
long	getKeepAliveTime(TimeUnit timeUnit)
protected long	getLastProcId()
<T extends Procedure<TEnvironment>> T	getProcedure(Class<T> clazz, long procId)
Procedure<TEnvironment>	getProcedure(long procId)
List<Procedure<TEnvironment>>	getProcedures() Get procedures.
Procedure<TEnvironment>	getResult(long procId)
Procedure<TEnvironment>	getResultOrProcedure(long procId)
ProcedureStore	getStore()
int	getUrgentPoolSize()
int	getWorkerThreadCount()
void	init(int numThreads, boolean abortOnCorruption) Initialize the procedure executor, but do not start workers.
void	init(int numThreads, int urgentNumThreads, boolean abortOnCorruption) Initialize the procedure executor, but do not start workers.
boolean	isFinished(long procId) Return true if the procedure is finished.
boolean	isProcedureOwner(long procId, User user) Check if the user is this procedure's owner
boolean	isRunning()
boolean	isStarted(long procId) Return true if the procedure is started.
void	join()
void	refreshConfiguration(org.apache.hadoop.conf.Configuration conf)
void	registerListener(ProcedureExecutor.ProcedureExecutorListener listener)
long	registerNonce(NonceKey nonceKey) Register a nonce for a procedure that is going to be submitted.
boolean	removeChore(ProcedureInMemoryChore<TEnvironment> chore) Remove a chore procedure from the executor
void	removeResult(long procId) Mark the specified completed procedure, as ready to remove.
void	setFailureResultForNonce(NonceKey nonceKey, String procName, User procOwner, IOException exception) If the failure failed before submitting it, we may want to give back the same error to the requests with the same nonceKey.
void	setKeepAliveTime(long keepAliveTime, TimeUnit timeUnit)
void	startWorkers() Start the workers.
void	stop()
long	submitProcedure(Procedure<TEnvironment> proc) Add a new root-procedure to the executor.
long	submitProcedure(Procedure<TEnvironment> proc, NonceKey nonceKey) Add a new root-procedure to the executor.
void	submitProcedures(Procedure<TEnvironment>[] procs) Add a set of new root-procedure to the executor.
boolean	unregisterListener(ProcedureExecutor.ProcedureExecutorListener listener)
void	unregisterNonceIfProcedureWasNotSubmitted(NonceKey nonceKey) Remove the NonceKey if the procedure was not submitted to the executor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CHECK_OWNER_SET_CONF_KEY

public static final String CHECK_OWNER_SET_CONF_KEY

See Also:

Constant Field Values

WORKER_KEEP_ALIVE_TIME_CONF_KEY

public static final String WORKER_KEEP_ALIVE_TIME_CONF_KEY

See Also:

Constant Field Values

UPGRADE_TO_2_2

public static final String UPGRADE_TO_2_2

See Also:

Constant Field Values

EVICT_TTL_CONF_KEY

public static final String EVICT_TTL_CONF_KEY

See Also:

Constant Field Values

EVICT_ACKED_TTL_CONF_KEY

public static final String EVICT_ACKED_TTL_CONF_KEY

See Also:

Constant Field Values

Constructor Detail

ProcedureExecutor

public ProcedureExecutor(org.apache.hadoop.conf.Configuration conf,
                         TEnvironment environment,
                         ProcedureStore store)

ProcedureExecutor

public ProcedureExecutor(org.apache.hadoop.conf.Configuration conf,
                         TEnvironment environment,
                         ProcedureStore store,
                         ProcedureScheduler scheduler)

Method Detail

init

public void init(int numThreads,
                 boolean abortOnCorruption)
          throws IOException

Initialize the procedure executor, but do not start workers. We will start them later.

It calls ProcedureStore.recoverLease() and ProcedureStore.load() to recover the lease, and ensure a single executor, and start the procedure replay to resume and recover the previous pending and in-progress procedures.

Parameters:

numThreads - number of threads available for procedure execution.

abortOnCorruption - true if you want to abort your service in case a corrupted procedure is found on replay. otherwise false.

Throws:

IOException

init

public void init(int numThreads,
                 int urgentNumThreads,
                 boolean abortOnCorruption)
          throws IOException

Initialize the procedure executor, but do not start workers. We will start them later.

It calls ProcedureStore.recoverLease() and ProcedureStore.load() to recover the lease, and ensure a single executor, and start the procedure replay to resume and recover the previous pending and in-progress procedures.

Parameters:

numThreads - number of threads available for procedure execution.

urgentNumThreads - number of threads available for urgent procedure execution.

abortOnCorruption - true if you want to abort your service in case a corrupted procedure is found on replay. otherwise false.

Throws:

IOException

startWorkers

public void startWorkers()
                  throws IOException

Start the workers.

Throws:

IOException

stop

public void stop()

join

public void join()

refreshConfiguration

public void refreshConfiguration(org.apache.hadoop.conf.Configuration conf)

isRunning

public boolean isRunning()

getWorkerThreadCount

public int getWorkerThreadCount()

Returns:

the current number of worker threads.

getCorePoolSize

public int getCorePoolSize()

Returns:

the core pool size settings.

getUrgentPoolSize

public int getUrgentPoolSize()

getActiveExecutorCount

public int getActiveExecutorCount()

getEnvironment

public TEnvironment getEnvironment()

getStore

public ProcedureStore getStore()

setKeepAliveTime

public void setKeepAliveTime(long keepAliveTime,
                             TimeUnit timeUnit)

getKeepAliveTime

public long getKeepAliveTime(TimeUnit timeUnit)

addChore

public void addChore(ProcedureInMemoryChore<TEnvironment> chore)

Add a chore procedure to the executor

Parameters:

chore - the chore to add

removeChore

public boolean removeChore(ProcedureInMemoryChore<TEnvironment> chore)

Remove a chore procedure from the executor

Parameters:

chore - the chore to remove

Returns:

whether the chore is removed, or it will be removed later

createNonceKey

public NonceKey createNonceKey(long nonceGroup,
                               long nonce)

Create a NoneKey from the specified nonceGroup and nonce.

Parameters:

nonceGroup -

nonce -

Returns:

the generated NonceKey

registerNonce

public long registerNonce(NonceKey nonceKey)

Register a nonce for a procedure that is going to be submitted. A procId will be reserved and on submitProcedure(), the procedure with the specified nonce will take the reserved ProcId. If someone already reserved the nonce, this method will return the procId reserved, otherwise an invalid procId will be returned. and the caller should procede and submit the procedure.

Parameters:

nonceKey - A unique identifier for this operation from the client or process.

Returns:

the procId associated with the nonce, if any otherwise an invalid procId.

unregisterNonceIfProcedureWasNotSubmitted

public void unregisterNonceIfProcedureWasNotSubmitted(NonceKey nonceKey)

Remove the NonceKey if the procedure was not submitted to the executor.

Parameters:

nonceKey - A unique identifier for this operation from the client or process.

setFailureResultForNonce

public void setFailureResultForNonce(NonceKey nonceKey,
                                     String procName,
                                     User procOwner,
                                     IOException exception)

If the failure failed before submitting it, we may want to give back the same error to the requests with the same nonceKey.

Parameters:

nonceKey - A unique identifier for this operation from the client or process

procName - name of the procedure, used to inform the user

procOwner - name of the owner of the procedure, used to inform the user

exception - the failure to report to the user

submitProcedure

public long submitProcedure(Procedure<TEnvironment> proc)

Add a new root-procedure to the executor.

Parameters:

proc - the new procedure to execute.

Returns:

the procedure id, that can be used to monitor the operation

bypassProcedure

public List<Boolean> bypassProcedure(List<Long> pids,
                                     long lockWait,
                                     boolean force,
                                     boolean recursive)
                              throws IOException

Bypass a procedure. If the procedure is set to bypass, all the logic in execute/rollback will be ignored and it will return success, whatever. It is used to recover buggy stuck procedures, releasing the lock resources and letting other procedures run. Bypassing one procedure (and its ancestors will be bypassed automatically) may leave the cluster in a middle state, e.g. region not assigned, or some hdfs files left behind. After getting rid of those stuck procedures, the operators may have to do some clean up on hdfs or schedule some assign procedures to let region online. DO AT YOUR OWN RISK.

A procedure can be bypassed only if 1. The procedure is in state of RUNNABLE, WAITING, WAITING_TIMEOUT or it is a root procedure without any child. 2. No other worker thread is executing it 3. No child procedure has been submitted

If all the requirements are meet, the procedure and its ancestors will be bypassed and persisted to WAL.

If the procedure is in WAITING state, will set it to RUNNABLE add it to run queue. TODO: What about WAITING_TIMEOUT?

Parameters:

pids - the procedure id

lockWait - time to wait lock

force - if force set to true, we will bypass the procedure even if it is executing. This is for procedures which can't break out during executing(due to bug, mostly) In this case, bypassing the procedure is not enough, since it is already stuck there. We need to restart the master after bypassing, and letting the problematic procedure to execute wth bypass=true, so in that condition, the procedure can be successfully bypassed.

recursive - We will do an expensive search for children of each pid. EXPENSIVE!

Returns:

true if bypass success

Throws:

IOException - IOException

submitProcedure

public long submitProcedure(Procedure<TEnvironment> proc,
                            NonceKey nonceKey)

Add a new root-procedure to the executor.

Parameters:

proc - the new procedure to execute.

nonceKey - the registered unique identifier for this operation from the client or process.

Returns:

the procedure id, that can be used to monitor the operation

submitProcedures

public void submitProcedures(Procedure<TEnvironment>[] procs)

Add a set of new root-procedure to the executor.

Parameters:

procs - the new procedures to execute.

abort

public boolean abort(long procId)

Send an abort notification the specified procedure. Depending on the procedure implementation the abort can be considered or ignored.

Parameters:

procId - the procedure to abort

Returns:

true if the procedure exists and has received the abort, otherwise false.

abort

public boolean abort(long procId,
                     boolean mayInterruptIfRunning)

Send an abort notification to the specified procedure. Depending on the procedure implementation, the abort can be considered or ignored.

Parameters:

procId - the procedure to abort

mayInterruptIfRunning - if the proc completed at least one step, should it be aborted?

Returns:

true if the procedure exists and has received the abort, otherwise false.

getProcedure

public Procedure<TEnvironment> getProcedure(long procId)

getProcedure

public <T extends Procedure<TEnvironment>> T getProcedure(Class<T> clazz,
                                                          long procId)

getResult

public Procedure<TEnvironment> getResult(long procId)

isFinished

public boolean isFinished(long procId)

Return true if the procedure is finished. The state may be "completed successfully" or "failed and rolledback". Use getResult() to check the state or get the result data.

Parameters:

procId - the ID of the procedure to check

Returns:

true if the procedure execution is finished, otherwise false.

isStarted

public boolean isStarted(long procId)

Return true if the procedure is started.

Parameters:

procId - the ID of the procedure to check

Returns:

true if the procedure execution is started, otherwise false.

removeResult

public void removeResult(long procId)

Mark the specified completed procedure, as ready to remove.

Parameters:

procId - the ID of the procedure to remove

getResultOrProcedure

public Procedure<TEnvironment> getResultOrProcedure(long procId)

isProcedureOwner

public boolean isProcedureOwner(long procId,
                                User user)

Check if the user is this procedure's owner

Parameters:

procId - the target procedure

user - the user

Returns:

true if the user is the owner of the procedure, false otherwise or the owner is unknown.

getActiveProceduresNoCopy

public Collection<Procedure<TEnvironment>> getActiveProceduresNoCopy()

Should only be used when starting up, where the procedure workers have not been started.

If the procedure works has been started, the return values maybe changed when you are processing it so usually this is not safe. Use getProcedures() below for most cases as it will do a copy, and also include the finished procedures.

getProcedures

public List<Procedure<TEnvironment>> getProcedures()

Get procedures.

Returns:

the procedures in a list

registerListener

public void registerListener(ProcedureExecutor.ProcedureExecutorListener listener)

unregisterListener

public boolean unregisterListener(ProcedureExecutor.ProcedureExecutorListener listener)

getLastProcId

protected long getLastProcId()

getActiveProcIds

public Set<Long> getActiveProcIds()