[maven-release-plugin] prepare release parquet-1.6.0rc1
[maven-release-plugin] prepare for next development iteration
PARQUET-73: Add support for FilterPredicates to cascading schemes

Author: Alex Levenson <alexlevenson@twitter.com>

Closes #34 from isnotinvain/alexlevenson/filter-cascading-scheme and squashes the following commits:

cd69a8e [Alex Levenson] Add support for FilterPredicates to cascading schemes
PARQUET-50: Re-Enable the semver enforcer

Author: Alex Levenson <alexlevenson@twitter.com>

Closes #37 from isnotinvain/alexlevenson/fix-backwards-compat-check and squashes the following commits:

c91335c [Alex Levenson] Bump version to 1.6.1 to make semver enforcer happy
c09c8c1 [Alex Levenson] PARQUET-50: Re-Enable the semver enforcer
Parquet-70: Fixed storing pig schema to udfcontext for non projection case and moved...

... column index access setting to udfcontext so as not to affect other loaders.

I found an problem that affects both the Column name access and column index access due to the way the pig schema is stored by the loader.

##Column Name Access:
The ParquetLoader was only storing the pig schema in the UDFContext when push projection is applied.  In the full load case, the schema was not stored which triggered a full reload of the schema during task execution.  You can see in initSchema references the UDFContext for the schema, but that is only set in push projection.  However, the schema needs to be set in both the job context (so the TupleReadSupport can access the schema) and the UDFContext (so the task side loader can access it), which is why it is set in both locations.  This also meant the requested schema was never set to the task side either, which could cause other problems as well.

##Column Index Access:
For index based access, the problem was that the column index access setting and the requested schema were not stored in the udfcontext and sent to the task side (unless pushProjection was called).  The schema was stored in the job context, but this would be overwritten if another loader was executed first.  Also, the property to use column index access was only being set at the job context level, so subsequent loaders would use column index access even if they didn't request it.

This fix now ensures that both the schema and column index access are set in the udfcontext and loaded in the initSchema method.

JIRA: https://issues.apache.org/jira/browse/PARQUET-70

-Dan

Author: Daniel Weeks <dweeks@netflix.com>

Closes #36 from dcw-netflix/pig-schema-context and squashes the following commits:

f896a25 [Daniel Weeks] Moved property loading into setInput
8f3dc28 [Daniel Weeks] Changed to set job conf settings in both front and backend
d758de0 [Daniel Weeks] Updated to use isFrontend() for setting context properties
b7ef96a [Daniel Weeks] Fixed storing pig schema to udfcontext for non projection case and moved column index access setting to udfcontext so as not to affect other loaders.
PARQUET-69: Committer doc

In order to improve turnaround time on reviewing pull requests here are a few guidelines and informations.

Author: Julien Le Dem <julien@twitter.com>

Closes #23 from julienledem/committer_doc and squashes the following commits:

d4b2e19 [Julien Le Dem] Update REVIEWERS.md
f7d8a5e [Julien Le Dem] Update COMMITTERS.md
b627d50 [Julien Le Dem] Create REVIEWERS.md
0027a98 [Julien Le Dem] Create REVIEWERS.md
6fd49d0 [Julien Le Dem] Create REVIEWERS.md
c6c26e6 [Julien Le Dem] Create REVIEWERS.md
cbbec60 [Julien Le Dem] Create REVIEWERS.md
56b7d3d [Julien Le Dem] Create REVIEWERS.md
a93ddcb [Julien Le Dem] Create REVIEWERS.md
98f1e52 [Julien Le Dem] Create REVIEWERS.md
5d960f4 [Julien Le Dem] Update COMMITTERS.md
7be5f36 [Julien Le Dem] Create REVIEWERS.md
24ef829 [Julien Le Dem] Create REVIEWERS.md
799a067 [Julien Le Dem] Create REVIEWERS.md
8ef1250 [Julien Le Dem] Create REVIEWERS.md
b85f454 [Julien Le Dem] Create REVIEWERS.md
38d1afa [Julien Le Dem] Update COMMITTERS.md
425224d [Julien Le Dem] Update COMMITTERS.md
46249ed [Julien Le Dem] Update COMMITTERS.md
7879de2 [Julien Le Dem] Create COMMITTERS.md
PARQUET-62: Fix binary dictionary write bug.

The binary dictionary writers keep track of written values in memory to
deduplicate and write dictionary pages periodically. If the written
values are changed by the caller, then this corrupts the dictionary
without an error message. This adds a defensive copy to fix the problem.

Author: Ryan Blue <rblue@cloudera.com>

Closes #29 from rdblue/PARQUET-62-fix-dictionary-bug and squashes the following commits:

42b6920 [Ryan Blue] PARQUET-62: Fix binary dictionary write bug.
PARQUET-67: mechanism to add extra metadata in the footer

this expands on the idea proposed by @wesleypeck in https://github.com/Parquet/parquet-mr/pull/185

Author: julien <julien@twitter.com>

Closes #32 from julienledem/extensible_metadata and squashes the following commits:

72e0a50 [julien] mechanism to add extra metadata in the footer
PARQUET-66: Upcast blockSize to long to prevent integer overflow.

Author: Eric Snyder <snyderep@gmail.com>

Closes #33 from snyderep/master and squashes the following commits:

c99802e [Eric Snyder] PARQUET-66: Upcast blockSize to long to prevent integer overflow.
PARQUET-80: upgrade semver plugin version to 0.9.27

To include the fix in:
https://github.com/jeluard/semantic-versioning/pull/39

Author: julien <julien@twitter.com>

Closes #46 from julienledem/upgrade_semver_plugin and squashes the following commits:

30e7247 [julien] upgrade semver plugin version to 0.9.27
PARQUET-75: Fixed string decode performance issue

Switch to using 'UTF8.decode' as opposed to 'new String'

https://issues.apache.org/jira/browse/PARQUET-75

Author: Daniel Weeks <dweeks@netflix.com>

Closes #40 from dcw-netflix/string-decode and squashes the following commits:

2cf53e7 [Daniel Weeks] Fixed string decode performance issue
PARQUET-8: bump scrooge-maven-plugin version

This makes eclipse:eclipse target work.

Author: dvryaboy <dvryaboy@gmail.com>
Author: Dmitriy Ryaboy <dmitriy@twitter.com>

Closes #41 from dvryaboy/PARQUET-8 and squashes the following commits:

f089bac [dvryaboy] trivial change to retrigger travis build.
d06a98e [Dmitriy Ryaboy] bump scrooge-maven-plugin version
do ProtocolEvents fixing only when there is required fields missing in the requested schema

https://issues.apache.org/jira/browse/PARQUET-61
This PR is trying to redo the https://github.com/apache/incubator-parquet-mr/pull/7

In this PR, it fixes the protocol event in a more precise condition:
Only when the requested schema missing some required fields that are present in the full schema

So even if there a projection, as long as the projection is not getting rid of the required field, the protocol events amender will not be called.

Could you take a look at this ? @dvryaboy @yan-qi

Author: Tianshuo Deng <tdeng@twitter.com>

Closes #28 from tsdeng/fix_protocol_when_required_field_missing and squashes the following commits:

ba778b9 [Tianshuo Deng] add continue for readability
d5639df [Tianshuo Deng] fix unused import
090e894 [Tianshuo Deng] format
13a609d [Tianshuo Deng] comment format
ef1fe58 [Tianshuo Deng] little refactor, remove the hasMissingRequiredFieldFromProjection method
7c2c158 [Tianshuo Deng] format
83a5655 [Tianshuo Deng] do ProtocolEvents fixing only when there is required fields missing in the requested schema
PARQUET-63: Enable dictionary encoding for FIXED.

This uses the existing dictionary support introduced for int96. Encoding
and ParquetProperties have been updated to use the dictionary supporting
classes, when requested for write or present during read. This also
fixes a bug in the fixed dictionary values writer, where the length was
hard-coded for int96, 12 bytes.

Author: Ryan Blue <rblue@cloudera.com>

Closes #30 from rdblue/PARQUET-63-add-fixed-dictionary-support and squashes the following commits:

bc34a34 [Ryan Blue] PARQUET-63: Enable dictionary encoding for FIXED.