org.apache.pig.tools.pigstats

Class PigStats

java.lang.Object

org.apache.pig.tools.pigstats.PigStats

Direct Known Subclasses:

EmptyPigStats, SimplePigStats, SparkPigStats, TezPigScriptStats

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public abstract class PigStats
extends java.lang.Object

PigStats encapsulates the statistics collected from a running script. It includes status of the execution, the DAG of its Hadoop jobs, as well as information about outputs and inputs of the script.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PigStats.JobGraph JobGraph is an OperatorPlan whose members are JobStats
static class	PigStats.JobGraphPrinter This class prints a JobGraph

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Map<java.lang.String,OutputStats>	aliasOuputMap
protected static java.lang.String	DATE_FORMAT
protected long	endTime
protected int	errorCode
protected java.lang.String	errorMessage
protected java.lang.Throwable	errorThrowable
protected PigStats.JobGraph	jobPlan
protected PigContext	pigContext
protected int	returnCode
protected long	startTime
protected java.lang.String	userId

Constructor Summary

Constructors

Constructor and Description
PigStats()

Method Summary

Modifier and Type	Method and Description
static PigStats	get()
abstract java.util.List<java.lang.String>	getAllErrorMessages()
abstract java.util.Map<java.lang.String,java.util.List<PigStats>>	getAllStats()
long	getBytesWritten() Returns the total bytes written to user specified HDFS locations of this script.
abstract java.lang.String	getDisplayString() Returns the display message in pig grunt
long	getDuration()
int	getErrorCode() Returns the error code of PigException
java.lang.String	getErrorMessage() Returns error message string
java.lang.Throwable	getErrorThrowable() Returns the error code of PigException
java.lang.String	getFeatures()
java.lang.String	getFileName()
java.lang.String	getHadoopVersion()
java.util.List<InputStats>	getInputStats()
abstract org.apache.hadoop.mapred.JobClient	getJobClient() Deprecated.
PigStats.JobGraph	getJobGraph() Returns the DAG of jobs spawned by the script
long	getNumberBytes(java.lang.String location) Returns the number of bytes for the given output location, -1 for invalid location or name.
int	getNumberFailedJobs()
int	getNumberJobs() Returns the number of jobs for this script
long	getNumberRecords(java.lang.String location) Returns the number of records for the given output location, -1 for invalid location or name.
int	getNumberSuccessfulJobs()
java.lang.String	getOutputAlias(java.lang.String location) Returns the alias associated with this output location
java.util.List<java.lang.String>	getOutputLocations() Returns the list of output locations in the script
java.util.List<java.lang.String>	getOutputNames() Returns the list of output names in the script
java.util.List<OutputStats>	getOutputStats()
PigContext	getPigContext()
java.util.Properties	getPigProperties() Returns the properties associated with the script
java.lang.String	getPigVersion()
abstract long	getProactiveSpillCountObjects() Returns the total number of bags that spilled proactively
abstract long	getProactiveSpillCountRecords() Returns the total number of records that spilled proactively
long	getRecordWritten() Returns the total number of records in user specified output locations of this script.
int	getReturnCode() Returns code are defined in PigRunner.ReturnCode
java.lang.String	getScript() Returns the contents of the script that was run.
java.lang.String	getScriptId()
abstract long	getSMMSpillCount() Returns the total spill counts from SpillableMemoryManager.
abstract boolean	isEmbedded()
boolean	isSuccessful()
OutputStats	result(java.lang.String alias)
void	setBackendException(java.lang.String jobId, java.lang.Exception e)
void	setErrorCode(int errorCode)
void	setErrorMessage(java.lang.String errorMessage)
void	setErrorThrowable(java.lang.Throwable t)
void	setReturnCode(int returnCode)
void	start()
static PigStats	start(PigStats stats)
void	stop()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DATE_FORMAT

protected static final java.lang.String DATE_FORMAT

See Also:

Constant Field Values

startTime

protected long startTime

endTime

protected long endTime

userId

protected java.lang.String userId

jobPlan

protected PigStats.JobGraph jobPlan

pigContext

protected PigContext pigContext

aliasOuputMap

protected java.util.Map<java.lang.String,OutputStats> aliasOuputMap

errorCode

protected int errorCode

errorMessage

protected java.lang.String errorMessage

errorThrowable

protected java.lang.Throwable errorThrowable

returnCode

protected int returnCode

Constructor Detail

PigStats

public PigStats()

Method Detail

get

public static PigStats get()

start

public static PigStats start(PigStats stats)

getReturnCode

public int getReturnCode()

Returns code are defined in PigRunner.ReturnCode

getErrorMessage

public java.lang.String getErrorMessage()

Returns error message string

getErrorCode

public int getErrorCode()

Returns the error code of PigException

getErrorThrowable

public java.lang.Throwable getErrorThrowable()

Returns the error code of PigException

getJobClient

@Deprecated
public abstract org.apache.hadoop.mapred.JobClient getJobClient()

Deprecated.

isEmbedded

public abstract boolean isEmbedded()

isSuccessful

public boolean isSuccessful()

getAllStats

public abstract java.util.Map<java.lang.String,java.util.List<PigStats>> getAllStats()

getAllErrorMessages

public abstract java.util.List<java.lang.String> getAllErrorMessages()

getPigProperties

public java.util.Properties getPigProperties()

Returns the properties associated with the script

getDisplayString

public abstract java.lang.String getDisplayString()

Returns the display message in pig grunt

getJobGraph

public PigStats.JobGraph getJobGraph()

Returns the DAG of jobs spawned by the script

getOutputLocations

public java.util.List<java.lang.String> getOutputLocations()

Returns the list of output locations in the script

getOutputNames

public java.util.List<java.lang.String> getOutputNames()

Returns the list of output names in the script

getNumberBytes

public long getNumberBytes(java.lang.String location)

Returns the number of bytes for the given output location, -1 for invalid location or name.

getNumberRecords

public long getNumberRecords(java.lang.String location)

Returns the number of records for the given output location, -1 for invalid location or name.

getOutputAlias

public java.lang.String getOutputAlias(java.lang.String location)

Returns the alias associated with this output location

getSMMSpillCount

public abstract long getSMMSpillCount()

Returns the total spill counts from SpillableMemoryManager.

getProactiveSpillCountObjects

public abstract long getProactiveSpillCountObjects()

Returns the total number of bags that spilled proactively

getProactiveSpillCountRecords

public abstract long getProactiveSpillCountRecords()

Returns the total number of records that spilled proactively

getBytesWritten

public long getBytesWritten()

Returns the total bytes written to user specified HDFS locations of this script.

getRecordWritten

public long getRecordWritten()

Returns the total number of records in user specified output locations of this script.

getHadoopVersion

public java.lang.String getHadoopVersion()

getPigVersion

public java.lang.String getPigVersion()

getScript

public java.lang.String getScript()

Returns the contents of the script that was run.

getScriptId

public java.lang.String getScriptId()

getFileName

public java.lang.String getFileName()

getFeatures

public java.lang.String getFeatures()

getDuration

public long getDuration()

getNumberJobs

public int getNumberJobs()

Returns the number of jobs for this script

getOutputStats

public java.util.List<OutputStats> getOutputStats()

result

public OutputStats result(java.lang.String alias)

getInputStats

public java.util.List<InputStats> getInputStats()

setErrorMessage

public void setErrorMessage(java.lang.String errorMessage)

setErrorCode

public void setErrorCode(int errorCode)

setErrorThrowable

public void setErrorThrowable(java.lang.Throwable t)

setReturnCode

public void setReturnCode(int returnCode)

setBackendException

@InterfaceAudience.Private
public void setBackendException(java.lang.String jobId,
                                                           java.lang.Exception e)

getPigContext

@InterfaceAudience.Private
public PigContext getPigContext()

start

public void start()

stop

public void stop()

getNumberSuccessfulJobs

public int getNumberSuccessfulJobs()

getNumberFailedJobs

public int getNumberFailedJobs()