com.splout.db.hadoop
Class TupleSQLite4JavaOutputFormat
java.lang.Object
org.apache.hadoop.mapreduce.OutputFormat<K,V>
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<com.datasalt.pangool.io.ITuple,org.apache.hadoop.io.NullWritable>
com.splout.db.hadoop.TupleSQLite4JavaOutputFormat
- All Implemented Interfaces:
- java.io.Serializable
public class TupleSQLite4JavaOutputFormat
- extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<com.datasalt.pangool.io.ITuple,org.apache.hadoop.io.NullWritable>
- implements java.io.Serializable
An OutputFormat that accepts Pangool's Tuples and writes to a sqlite4Java SQLite file. The Tuples that are written to
it must conform to a particular schema: having a "_partition" integer field (which will then create a file named
"partition".db).
The different schemas that will be given to this OutputFormat are defined in the constructor by providing a
TableSpec
. These TableSpec also contains information such as pre-SQL or post-SQL statements but most notably
contain a Schema so that a CREATE TABLE can be derived automatically from it. Note that the Schema provided to
TableSpec doesn't need to contain a "_partition" field or be nullable.
- See Also:
- Serialized Form
Method Summary |
protected static java.lang.String[] |
getCreateIndexes(TableSpec... tableSpecs)
|
protected static java.lang.String[] |
getCreateTables(TableSpec... tableSpecs)
|
org.apache.hadoop.mapreduce.RecordWriter<com.datasalt.pangool.io.ITuple,org.apache.hadoop.io.NullWritable> |
getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
|
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat |
checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
PARTITION_TUPLE_FIELD
public static final java.lang.String PARTITION_TUPLE_FIELD
- See Also:
- Constant Field Values
LOG
public static org.apache.commons.logging.Log LOG
TupleSQLite4JavaOutputFormat
public TupleSQLite4JavaOutputFormat(int batchSize,
TableSpec... dbSpec)
throws TupleSQLite4JavaOutputFormat.TupleSQLiteOutputFormatException
- This OutputFormat receives a list of
TableSpec
. These are the different tables that will be created. They
will be identified by Pangool Tuples. The batch size is the number of SQL statements to execute before a COMMIT.
- Throws:
TupleSQLite4JavaOutputFormat.TupleSQLiteOutputFormatException
getCreateTables
protected static java.lang.String[] getCreateTables(TableSpec... tableSpecs)
throws TupleSQLite4JavaOutputFormat.TupleSQLiteOutputFormatException
- Throws:
TupleSQLite4JavaOutputFormat.TupleSQLiteOutputFormatException
getCreateIndexes
protected static java.lang.String[] getCreateIndexes(TableSpec... tableSpecs)
throws TupleSQLite4JavaOutputFormat.TupleSQLiteOutputFormatException
- Throws:
TupleSQLite4JavaOutputFormat.TupleSQLiteOutputFormatException
getRecordWriter
public org.apache.hadoop.mapreduce.RecordWriter<com.datasalt.pangool.io.ITuple,org.apache.hadoop.io.NullWritable> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
throws java.io.IOException,
java.lang.InterruptedException
- Specified by:
getRecordWriter
in class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<com.datasalt.pangool.io.ITuple,org.apache.hadoop.io.NullWritable>
- Throws:
java.io.IOException
java.lang.InterruptedException
Copyright © 2012-2013 Datasalt Systems S.L.. All Rights Reserved.