PARQUET-84: Avoid reading rowgroup metadata in memory on the client side.

This will improve reading big datasets with a large schema (thousands of columns)
Instead rowgroup metadata can be read in the tasks where each tasks reads only the metadata of the file it's reading

Author: julien <julien@twitter.com>

Closes #45 from julienledem/skip_reading_row_groups and squashes the following commits:

ccdd08c [julien] fix parquet-hive
24a2050 [julien] Merge branch 'master' into skip_reading_row_groups
3d7e35a [julien] adress review feedback
5b6bd1b [julien] more tests
323d254 [julien] sdd unit tests
f599259 [julien] review feedback
fb11f02 [julien] fix backward compatibility check
2c20b46 [julien] cleanup readFooters methods
3da37d8 [julien] fix read summary
ab95a45 [julien] cleanup
4d16df3 [julien] implement task side metadata
9bb8059 [julien] first stab at integrating skipping row groups
update scala 2.10

Try to upgrade to scala 2.10

Author: Tianshuo Deng <tdeng@twitter.com>

Closes #35 from tsdeng/update_scala_2_10 and squashes the following commits:

1b7e55f [Tianshuo Deng] fix comment
bed9de3 [Tianshuo Deng] remove twitter artifactory
2bce643 [Tianshuo Deng] publish fix
06b374e [Tianshuo Deng] define scala.binary.version
fcf6965 [Tianshuo Deng] Merge branch 'master' into update_scala_2_10
e91d9f7 [Tianshuo Deng] update version
5d18b88 [Tianshuo Deng] version
83df898 [Tianshuo Deng] update scala 2.10
upgrade scalatest_version to depend on scala 2.10.4

Author: julien <julien@twitter.com>

Closes #52 from julienledem/scalatest_version and squashes the following commits:

945fa75 [julien] upgrade scalatest_version to depend on scala 2.10.4
PARQUET-87: Add API for projection pushdown on the cascading scheme level

JIRA: https://issues.apache.org/jira/browse/PARQUET-87
Previously, the projection pushdown configuration is global, and not bind to a specific tap.
After adding this API, projection pushdown can be done more "naturally", which may benefit scalding. The code that uses this API would look like:

```
Scheme sourceScheme = new ParquetScroogeScheme(new Config().withProjection(projectionFilter));
 Tap source = new Hfs(sourceScheme, PARQUET_PATH);
```

Author: Tianshuo Deng <tdeng@twitter.com>

Closes #51 from tsdeng/projection_from_scheme and squashes the following commits:

2c72757 [Tianshuo Deng] make config class final
813dc1a [Tianshuo Deng] erge branch 'master' into projection_from_scheme
b587b79 [Tianshuo Deng] make constructor of Config private, fix format
3aa7dd2 [Tianshuo Deng] remove builder
9348266 [Tianshuo Deng] use builder()
7c91869 [Tianshuo Deng] make fields of Config private, create builder method for Config
5fdc881 [Tianshuo Deng] builder for setting projection pushdown and predicate pushdown
a47f271 [Tianshuo Deng] immutable
3d514b1 [Tianshuo Deng] done
PARQUET-88: fix pre-version enforcement

update semver dependency to use https://github.com/jeluard/semantic-versioning/pull/40

Author: julien <julien@twitter.com>

Closes #53 from julienledem/fix_semver and squashes the following commits:

70248be [julien] fix pre-version enforcement https://github.com/jeluard/semantic-versioning/pull/40
PARQUET-24: enforce JIRA prefix

Author: julien <julien@twitter.com>

Closes #54 from julienledem/PARQUET-24 and squashes the following commits:

d444f2f [julien] add option to not close JIRA
00d4b13 [julien] fix close jira
7cf9bb3 [julien] enforce JIRA prefix
[maven-release-plugin] prepare release parquet-1.6.0rc2
[maven-release-plugin] prepare for next development iteration
PARQUET-94: Fix bug in ParquetScroogeScheme constructor, minor cleanup

I noticed that ParquetScroogeScheme's constructor ignores the provided klass argument.
I also added in missing type parameters for the Config object where they were missing.

Author: Alex Levenson <alexlevenson@twitter.com>

Closes #61 from isnotinvain/alexlevenson/parquet-scrooge-cleanup and squashes the following commits:

2b16007 [Alex Levenson] Fix bug in ParquetScroogeScheme constructor, minor cleanup
PARQUET-92: Pig parallel control

The parallelism for reading footers was fixed at '5', which isn't optimal for using pig with S3.  Just adding a property to adjust the parallelism.

JIRA: https://issues.apache.org/jira/browse/PARQUET-92

Author: Daniel Weeks <dweeks@netflix.com>

Closes #57 from dcw-netflix/pig-parallel-control and squashes the following commits:

e49087c [Daniel Weeks] Update ParquetFileReader.java
ec4f8ca [Daniel Weeks] Added configurable control of parallelism
d37a6de [Daniel Weeks] Resetting pom to main
0c1572e [Daniel Weeks] Merge remote-tracking branch 'upstream/master'
98c6607 [Daniel Weeks] Merge remote-tracking branch 'upstream/master'
96ba602 [Daniel Weeks] Disabled projects that don't compile
PARQUET-89: Add hadoop-2 test profile for Travis CI.

This also fixes problems that prevented hadoop-2 from passing:
* Dynamically resolve counter methods in parquet-hadoop
* Parameterize pig version with hadoop-2 support
* Update pig test for hadoop-2 change (no nulls allowed)
* Update parquet-hive to depend on hadoop-client

The travis config will now run each test profile in a different run.

Author: Ryan Blue <rblue@cloudera.com>

Closes #55 from rdblue/PARQUET-89-add-test-profiles and squashes the following commits:

006c6d8 [Ryan Blue] PARQUET-89: Add hadoop-2 test profile for travis CI.
PARQUET-82: Check page size is valid when writing.

Author: Ryan Blue <rblue@cloudera.com>

Closes #48 from rdblue/PARQUET-82-check-page-size and squashes the following commits:

9f31402 [Ryan Blue] PARQUET-82: Check page size is valid when writing.
PARQUET-101: fix meta data lookup when not using task.side.metadata

Author: julien <julien@twitter.com>

Closes #64 from julienledem/PARQUET-101 and squashes the following commits:

54ffbc9 [julien] fix meta data lookup when not using task.side.metadata
PARQUET-90: integrate field ids in schema

This integrates support for field is that was introduced in Parquet format.
Thrift and Protobufs ids will now be saved in the Parquet schema.

Author: julien <julien@twitter.com>

Closes #56 from julienledem/field_ids and squashes the following commits:

62c2809 [julien] remove withOriginalType; use Typles builder more
8ff0034 [julien] review feedback
084c8be [julien] binary compat
85d785c [julien] add proto id in schema; fix schema parsing for ids
d4be488 [julien] integrate field ids in schema
PARQUET-96: fill out some missing methods on parquet.example classes

I'm slightly embarrassed to say that we use these, and we'd really like to stop needing a fork, so here we are.

Author: Colin Marc <colinmarc@gmail.com>

Closes #59 from colinmarc/missing-group-methods and squashes the following commits:

af8ea08 [Colin Marc] fill out some missing methods on parquet.example classes
PARQUET-104: Fix writing empty row group at the end of the file

At then end of a parquet file, it may writes an empty rowgroup.
This happens when: numberOfRecords mod sizeOfRowGroup = 0

Author: Tianshuo Deng <tdeng@twitter.com>

Closes #66 from tsdeng/fix_empty_row_group and squashes the following commits:

10b93fb [Tianshuo Deng] rename
e3a5896 [Tianshuo Deng] format
91fa0d4 [Tianshuo Deng] fix empty row group
PARQUET-64: Add new OriginalTypes in parquet-format 2.2.0.

This implements the restrictions for those types documented in the parquet-format logical types spec.

This requires a release of parquet-format 2.2.0 with the new types. I'll rebase and update the dependency when it is released.

Author: Ryan Blue <rblue@cloudera.com>

Closes #31 from rdblue/PARQUET-64-add-new-types and squashes the following commits:

10feab9 [Ryan Blue] PARQUET-64: Add new OriginalTypes in parquet-format 2.2.0.
PARQUET-107: Add option to disable summary metadata.

This adds an option to the commitJob phase of the MR OutputCommitter,
parquet.enable.summary-metadata (default true), that can be used to
disable the summary metadata files generated from the footers of all of
the files produced. This enables more control over when those summary
files are produced and makes it possible to rename MR outputs and then
generate the summaries.

Author: Ryan Blue <rblue@cloudera.com>

Closes #68 from rdblue/PARQUET-107-add-summary-metadata-option and squashes the following commits:

261e5e4 [Ryan Blue] PARQUET-107: Add option to disable summary metadata.