org.apache.pig.tools.pigstats.mapreduce

Class MRJobStats

java.lang.Object

org.apache.pig.newplan.Operator

org.apache.pig.tools.pigstats.JobStats

org.apache.pig.tools.pigstats.mapreduce.MRJobStats

All Implemented Interfaces:

java.io.Serializable

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public final class MRJobStats
extends JobStats

This class encapsulates the runtime statistics of a MapReduce job. Job statistics is collected when job is completed.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.tools.pigstats.JobStats

JobStats.JobState

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	FAILURE_HEADER
static java.lang.String	SUCCESS_HEADER

Fields inherited from class org.apache.pig.tools.pigstats.JobStats

ALIAS, ALIAS_LOCATION, conf, FEATURE, hdfsBytesRead, hdfsBytesWritten, inputs, outputs, state

Fields inherited from class org.apache.pig.newplan.Operator

annotations, hashPrime, location, name, plan

Method Summary

Modifier and Type	Method and Description
void	accept(PlanVisitor v) Accept a visitor at this node in the graph.
java.lang.String	getAlias()
java.lang.String	getAliasLocation()
long	getAvgMapTime()
long	getAvgREduceTime()
static org.apache.hadoop.mapred.Counters	getCounters(org.apache.hadoop.mapred.jobcontrol.Job job)
java.lang.String	getDisplayString()
java.lang.String	getFeature()
org.apache.hadoop.mapred.Counters	getHadoopCounters()
java.lang.String	getJobId()
long	getMapInputRecords()
long	getMapOutputRecords()
long	getMaxMapTime()
long	getMaxReduceTime()
long	getMinMapTime()
long	getMinReduceTime()
java.util.Map<java.lang.String,java.lang.Long>	getMultiInputCounters()
java.util.Map<java.lang.String,java.lang.Long>	getMultiStoreCounters()
int	getNumberMaps()
int	getNumberReduces()
long	getProactiveSpillCountObjects()
long	getProactiveSpillCountRecs()
long	getReduceInputRecords()
long	getReduceOutputRecords()
long	getSMMSpillCount()
static java.util.Iterator<org.apache.hadoop.mapreduce.TaskReport>	getTaskReports(org.apache.hadoop.mapred.jobcontrol.Job job, org.apache.hadoop.mapreduce.TaskType type)
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class org.apache.pig.tools.pigstats.JobStats

calculateMedianValue, getBytesWritten, getErrorMessage, getException, getHdfsBytesRead, getHdfsBytesWritten, getInputs, getOutputs, getOutputSize, getRecordWrittern, getState, isEqual, isIndexer, isSampler, isSuccessful, setBackendException, setErrorMsg, setSuccessful

Methods inherited from class org.apache.pig.newplan.Operator

annotate, getAnnotation, getLocation, getName, getPlan, removeAnnotation, setLocation, setPlan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SUCCESS_HEADER

public static final java.lang.String SUCCESS_HEADER

See Also:

Constant Field Values

FAILURE_HEADER

public static final java.lang.String FAILURE_HEADER

See Also:

Constant Field Values

Method Detail

getJobId

public java.lang.String getJobId()

Specified by:

getJobId in class JobStats

getNumberMaps

public int getNumberMaps()

Specified by:

getNumberMaps in class JobStats

getNumberReduces

public int getNumberReduces()

Specified by:

getNumberReduces in class JobStats

getMaxMapTime

public long getMaxMapTime()

Specified by:

getMaxMapTime in class JobStats

getMinMapTime

public long getMinMapTime()

Specified by:

getMinMapTime in class JobStats

getAvgMapTime

public long getAvgMapTime()

Specified by:

getAvgMapTime in class JobStats

getMaxReduceTime

public long getMaxReduceTime()

Specified by:

getMaxReduceTime in class JobStats

getMinReduceTime

public long getMinReduceTime()

Specified by:

getMinReduceTime in class JobStats

getAvgREduceTime

public long getAvgREduceTime()

Specified by:

getAvgREduceTime in class JobStats

getMapInputRecords

public long getMapInputRecords()

Specified by:

getMapInputRecords in class JobStats

getMapOutputRecords

public long getMapOutputRecords()

Specified by:

getMapOutputRecords in class JobStats

getReduceInputRecords

public long getReduceInputRecords()

Specified by:

getReduceInputRecords in class JobStats

getReduceOutputRecords

public long getReduceOutputRecords()

Specified by:

getReduceOutputRecords in class JobStats

getSMMSpillCount

public long getSMMSpillCount()

Specified by:

getSMMSpillCount in class JobStats

getProactiveSpillCountObjects

public long getProactiveSpillCountObjects()

Specified by:

getProactiveSpillCountObjects in class JobStats

getProactiveSpillCountRecs

public long getProactiveSpillCountRecs()

Specified by:

getProactiveSpillCountRecs in class JobStats

getHadoopCounters

public org.apache.hadoop.mapred.Counters getHadoopCounters()

Specified by:

getHadoopCounters in class JobStats

getMultiStoreCounters

public java.util.Map<java.lang.String,java.lang.Long> getMultiStoreCounters()

Specified by:

getMultiStoreCounters in class JobStats

getMultiInputCounters

public java.util.Map<java.lang.String,java.lang.Long> getMultiInputCounters()

Specified by:

getMultiInputCounters in class JobStats

getAlias

public java.lang.String getAlias()

Overrides:

getAlias in class JobStats

getAliasLocation

public java.lang.String getAliasLocation()

Overrides:

getAliasLocation in class JobStats

getFeature

public java.lang.String getFeature()

Overrides:

getFeature in class JobStats

accept

public void accept(PlanVisitor v)
            throws FrontendException

Description copied from class: Operator

Accept a visitor at this node in the graph.

Specified by:

accept in class JobStats

Parameters:

v - Visitor to accept.

Throws:

FrontendException

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Overrides:

setConf in class JobStats

getDisplayString

public java.lang.String getDisplayString()

Specified by:

getDisplayString in class JobStats

getTaskReports

public static java.util.Iterator<org.apache.hadoop.mapreduce.TaskReport> getTaskReports(org.apache.hadoop.mapred.jobcontrol.Job job,
                                                                                        org.apache.hadoop.mapreduce.TaskType type)
                                                                                 throws java.io.IOException

Throws:

java.io.IOException

getCounters

public static org.apache.hadoop.mapred.Counters getCounters(org.apache.hadoop.mapred.jobcontrol.Job job)
                                                     throws java.io.IOException

Throws:

java.io.IOException