Adding Type Persuasion for Primitive Types

Original from the old repo: https://github.com/Parquet/parquet-mr/pull/410
JIRA: https://issues.apache.org/jira/browse/PARQUET-2

These changes allow primitive types to be requested as different types than what is stored in the file format using a flag to turn off strict type checking (default is on). Types are cast to the requested type where possible and will suffer precision loss for casting where necessary (e.g. requesting a double as an int).

No performance penalty is imposed for using the type defined in the file type.  A flag exists to 

A 6x6 test case is provided to test conversion between the primitive types.

I looked into this one a bit and found that using the constructor results
two different property reads.

Currently, ParquetInputFormat and InternalParquetRecordReader read the
property, but if I take it out of InternalParquetRecordReader and change it
to a parameter, then ParquetReader will need to read the property as well
since it instantiates the InternalParquetRecordReader without being called
by the ParquetInputFormat.

I'm not sure there's a clean way to do this.

-Dan


On Fri, Jun 20, 2014 at 9:26 PM, Julien Le Dem <notifications@github.com>
wrote:

> In
> parquet-hadoop/src/main/java/parquet/hadoop/InternalParquetRecordReader.java:
>
> > @@ -106,7 +108,7 @@ private void checkRead() throws IOException {
> >        BenchmarkCounter.incrementTime(timeSpentReading);
> >        LOG.info("block read in memory in " + timeSpentReading + " ms. row count = " + pages.getRowCount());
> >        if (Log.DEBUG) LOG.debug("initializing Record assembly with requested schema " + requestedSchema);
> > -      MessageColumnIO columnIO = columnIOFactory.getColumnIO(requestedSchema, fileSchema);
> > +      MessageColumnIO columnIO = columnIOFactory.getColumnIO(requestedSchema, fileSchema, strictTypeChecking);
>
> the ParquetInputFormat already has read this setting from the conf.
> I'd rather keep this centralized there and passed as a constructor
> argument to the RecordReader.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/incubator-parquet-mr/pull/3/files#r14048288>.
>