Add SHOW DATA SOURCES

Change-Id: Ieeb0df107f45a58b8a99f717e96453da93ee7270
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2529
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: jenkins
IMPALA-994: Handle incorrect column metadata in views created by Hive.

Change-Id: I3fba08d191c479f37371ce50fd07b8476a73eba2
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2613
Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
Tested-by: jenkins
IMPALA-996: Exclude non-materialized slots from a tuple's avgSerializedSize.

Change-Id: Ic7936c6b5c5e6d4c162d91105128cda2b1b7284c
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2617
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
CDH-18512: Modification to allow spaces around the = sign in SET in impala-shell

Change-Id: I3c149e9a27962ed1130b1ddbb02952f4254bd4c9
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2609
Reviewed-by: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
Tested-by: jenkins
Cleanup builtins db initialization

Instead of doing the initialization in Catalog.java, there is now a special
BuiltinsDb that handles this initialization. This makes in more clear what file
should be modified when adding a new builtin and also cuts down the code in the Catalog.

Change-Id: I4512abff6e8c7f4924701aeffe10e656104a0b86
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2567
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins
Add TRole/TPrivilege structs to Thrift CatalogObjects

These are used as our internal representation of the authorization policy metadata
(as opposed to directly using the Sentry Thrift structs). Versioned/managed in the
same way as other TCatalogObjects.

Change-Id: Ia1ed9bd4e25e9072849edebcae7c2d3a7aed660d
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2545
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins
Address CR comments on HdfsAvroScanner.

Change-Id: Ia92b07e0b0cd16297c0d84f0df5e5eff76788c3e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2610
Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
Tested-by: jenkins
IMPALA-984: Fix missing reanalyze in InlineViewRef and NULL handling.

Change-Id: Ia80035c5456630aeef7a24288a998fe08546a282
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2628
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
Allow adding predicates after processing build table.

Change-Id: I4c845d9f08f0be29e548eceac3912871acd0270f
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2480
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
Add option to build UDF test libs in copy-udfs-udas.sh

The option is off by default, but useful for running this script
without building the world.

Change-Id: I82d8251cf9bb2763ce69094da1995a4d6ceff167
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2647
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
IMPALA-938: Fix implicit casting in timestamp arithmetic exprs.

Change-Id: I7e875ec2251e9782c98b60195ecbc92258b63b5c
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2657
Reviewed-by: Victor Bittorf <victor.bittorf@cloudera.com>
Tested-by: jenkins
IMPALA-825: Synchronize Hive Metastore client connection creation

The creation of hive metastore clients is now synchronised to avoid the possibility of
race conditions accessing local Kerberos state. In experiments, this does not fully
resolve the issue but significantly reduces the chances it will occur.

Also adds in a new debug config option to optionally sleep for a specified number of
milliseconds between creating connections. The default is zero, but can be configured
by setting impala.catalog.metastore.cnxn.creation.delay.ms in the hive-site.xml.

Change-Id: I83e863760470bdc2d9b27c6669f35604111d69d7
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2661
Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
Tested-by: jenkins
Fix comment in BitWriter::PutAligned from bits to bytes.

Change-Id: I22d029bc219ef49b94610d3e0ce9d9f2f4927bd6
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2670
Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
Tested-by: jenkins
CDH-19292: Impala query on an HBase table takes almost same time as Hive

This commit fixes issue CDH-19292, where querying an HBase table takes
a significant amount of time if HBase has a large (>1K) number of
tables. The performance bottleneck is the call to HBase to retrieve the
table's metadata (column families) during the computation of row stats.
To resolve this issue we cache the column families in the catalog object
associated with an HBase table, so the expensive call to HBase only
happens the first time the table is queried. Subsequent calls use the
cached data.

Change-Id: I0203fee3e73d2a4304530fe0a1ba2cf163f39350
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2672
Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins
IMPALA-1003: Speed up compute stats by avoiding null counting for all types and max/avg for fixed-size types.

Removing the Null counting as well as the MAX and AVG calculations
for the types of known size

Change-Id: If218b41fcde0b18218732675585d69342f34e544
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2629
Reviewed-by: Ippokratis Pandis <ipandis@cloudera.com>
Tested-by: jenkins
Disable hive UDFs that use decimal in analysis.

Change-Id: Ifd3e134d0e8ba6ad054e5769da57c9a830ad74d2
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2676
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
Move CodegenAnyVal to its own file.

This is needed by upcoming commits so we can #include anyval-util.h in
cross-compiled functions without pulling in all of LLVM in our IR
module (this causes build problems, plus our module will be huger than
necessary).

Change-Id: I756d5a95e5c254403d9ad5684fe27cf96f3aec1e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2677
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
IMPALA-887: Improve partition pruning time

This commit is the first step in improving the performance of partition
pruning. Currently, Impala can prune approximately 10K partitions per
sec, thereby introducing significant overhead for huge table with a
large number of partitions. With this commit we reduce that overhead by
3X by batching the partition pruning calls to the backend.

Change-Id: I3303bfc7fb6fe014790f58a5263adeea94d0fe7d
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2608
Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Tested-by: jenkins
The workload runner should abort execution is a query fails in a multi-user run.

Currently, we coalesce the results and do not properly catch a failure if one of the
threads has a failed query and exit_on_error is set to True. This patch ensures that we
exit before the next query is run.

Change-Id: Ie650e0f547874386c79c78982ea9916f33e18cda
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2696
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: jenkins
IMPALA-1011: Handle SHOW DATA SOURCES when no sources configured

Change-Id: I367b90c7603aea973d442f9186a6b32598a66a28
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2716
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins