org.apache.hadoop.hbase.mapreduce

Class MultiTableOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,Mutation>

org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat

@InterfaceAudience.Public
public class MultiTableOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,Mutation>

Hadoop output format that writes to one or more HBase tables. The key is taken to be the table name while the output value must be either a Put or a Delete instance. All tables must already exist, and all Puts and Deletes must reference only valid column families.

Write-ahead logging (WAL) for Puts can be disabled by setting WAL_PROPERTY to WAL_OFF. Default value is WAL_ON. Note that disabling write-ahead logging is only appropriate for jobs where loss of data due to region server failure can be tolerated (for example, because it is easy to rerun a bulk import).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	MultiTableOutputFormat.MultiTableRecordWriter Record writer for outputting to multiple HTables.

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	WAL_OFF Property value to disable write-ahead logging
static boolean	WAL_ON Property value to use write-ahead logging
static String	WAL_PROPERTY Set this to WAL_OFF to turn off write-ahead logging (WAL)

Constructor Summary

Constructors

Constructor and Description
MultiTableOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.hadoop.mapreduce.RecordWriter<ImmutableBytesWritable,Mutation>	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

WAL_PROPERTY

public static final String WAL_PROPERTY

Set this to WAL_OFF to turn off write-ahead logging (WAL)

See Also:

Constant Field Values

WAL_ON

public static final boolean WAL_ON

Property value to use write-ahead logging

See Also:

Constant Field Values

WAL_OFF

public static final boolean WAL_OFF

Property value to disable write-ahead logging

See Also:

Constant Field Values

Constructor Detail

MultiTableOutputFormat

public MultiTableOutputFormat()

Method Detail

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws IOException,
                             InterruptedException

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,Mutation>

Throws:

IOException

InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws IOException,
                                                                      InterruptedException

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,Mutation>

Throws:

IOException

InterruptedException

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<ImmutableBytesWritable,Mutation> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                          throws IOException,
                                                                                                 InterruptedException

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<ImmutableBytesWritable,Mutation>

Throws:

IOException

InterruptedException