The workload runner should abort execution is a query fails in a multi-user run.

Currently, we coalesce the results and do not properly catch a failure if one of the
threads has a failed query and exit_on_error is set to True. This patch ensures that we
exit before the next query is run.

Change-Id: Ie650e0f547874386c79c78982ea9916f33e18cda
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2696
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: jenkins
IMPALA-1011: Handle SHOW DATA SOURCES when no sources configured

Change-Id: I367b90c7603aea973d442f9186a6b32598a66a28
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2716
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
IMPALA-959: Fix ASAN decimal crashes.

Not quite sure what the underlying issue is but these fixes seem to work.

Change-Id: I759804eb8338ba86969c0214a1e6e35588c94297
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2698
Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
Tested-by: jenkins
IMPALA-1010: Remove Close() of build side in blocking join node.

This optimization is generally not safe since the probe side is still streaming. The
join node could acquire all of the data from the child into its own pool but then
there's no real point in doing this (doesn't lead to lower memory footprint and just
makes the mem accounting harder to reason about).

This is exposed in busy plans.

Change-Id: I37b0f6507dc67c79e5ebe8b9242ec86f28ddad41
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2722
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
IOMgr changes for order-by without limit

Change-Id: I361d1c6d0f588a726f2add3f96bd1fc724ed83ac
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2666
Reviewed-by: Srinath Shankar <sshankar@cloudera.com>
Tested-by: jenkins
Add decimal UDF/UDA support.

Change-Id: Ie48c1cb8e978c7282593b7f602dd68added6d3fd
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2625
Reviewed-by: Skye Wanderman-Milne <skye@cloudera.com>
Tested-by: jenkins
Fix recovering from Llama restart. Refactor connect/rpc/retry logic.

Change-Id: Idfbbfc3141cb9774d30ed1b1da4fe2cd9e511889
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2685
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2737
Add decimal builtins: abs/round/ceil/floor/truncate.

Change-Id: I4fe0ee69475ff56d3dc0cd69ea21f677714ae8bc
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2455
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
CDH-19348 LDAP + SSL

TSSLSocket should not be "opened" if it's used by the server. See TSSLSocket::open()

Therefore, in TSaslTransport::open(), it should not open underlying transport if it's a sever.

I've tested it manually with LDAP and LDAP+SSL, but we don't have functional test for LDAP yet.

Change-Id: Ifee4957c6a874df47760d33ab50aa90eb7eda617
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2718
Reviewed-by: Alan Choi <alan@cloudera.com>
Tested-by: jenkins
IMPALA-1016: Return correct number of NULL values when projecting newly added column

This patch handles the case where when a query was projecting a newly added column,
the parquet scanner was returning infinite values.

Change-Id: Ie5f4d4a88d5868e8d9e5c39fa9440821776dde3c
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2725
Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
Tested-by: jenkins
Detect pseudo-distributed Llama cluster

Since we're no longer using the MiniLlama, we need to explicitly set
whether or not the cluster is pseudo-distributed. Impala needs this
information to correctly translate datanode addresses to a format that
Llama understands.

This change (adapted from one made by Casey) adds a method to the
frontend (callable via JNI) to get a configuration value from the Hadoop
configuration. We'll set that configuration value for local RM testing.

Change-Id: Ifd51db98a993ac0270dac2b832babbc394483c1a
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2549
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2754
[CDH4] IMPALA-1019: Fix cancellation path in io mgr for cached reads.

Change-Id: I33eea3caffb1e687af8c3e6054aa48b97cc0db73
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2756
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
IMPALA-974: Switch from FloatLiteral to DecimalLiteral.

Float/Doubles are lossy so using those as the default literal type
is problematic.

Change-Id: I5a619dd931d576e2e6cd7774139e9bafb9452db9
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2615
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: Nong Li <nong@cloudera.com>
Support Llama HA by cycling through a list of configured Llama hostports.

Change-Id: I471388f468254598347fdf605669b0c0af0f7a15
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2707
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2760
Codegen NDV computation.

The HLL update and merge functions are cross-compiled and called in
the codegen'd UpdateSlot() function. (The UdfContext functions are
also cross-compiled so they can be inlined.) This speeds up NDV
calculation 2-3x.

Change-Id: Ia0de5e231e4520097ee1a4df8a3dfda5b1843738
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2732
Reviewed-by: Skye Wanderman-Milne <skye@cloudera.com>
Tested-by: jenkins
IMPALA-895: Don't log cancelled to error log.

Change-Id: I1d06c4dc94f59b413678589d663eb77724482d5e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2771
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
Fix decimal literal casting and cast expr reanalyze().

BigDecimal doesn't think about scale the way we need it to.

Change-Id: I09612c31e30e80ce4806080f1d24c6615090785e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2779
Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
Tested-by: jenkins
[CDH5->CDH4] Add DDL support for HDFS caching

This change adds DDL support for HDFS caching. The DDL allows the user to indicate a
table or partition should be cached and which pool to cache the data into:
* Create a cached table: CREATE TABLE ... CACHED IN 'poolName'
* Cache a table/partition: ALTER TABLE ... [partitionSpec] SET CACHED IN 'poolName'
* Uncache a table/partition: ALTER TABLE ... [partitionSpec] SET UNCACHED

When a table/partition is marked as cached, a new HDFS caching request is submitted
to cache the location (HDFS path) of the table/partition and the ID of that request
is stored with in the table metadata (in the table properties). This is stored as:
'cache_directive_id'='<requestId>'. The cache requests and IDs are managed by HDFS
and persisted across HDFS restarts.

When a cached table or partition is dropped it is important to uncache the cached data
(drop the associated cache request). For partitioned tables, this means dropping all
cache requests from all cached partitions in the table.
Likewise, if a partitioned table is created as cached, new partitions should be marked
as cached by default.

It is desirable to know which cache pools exists early on (in analysis) so the query
will fail without hitting HDFS/CatalogServer if a non-existent pool is specified. To
support this, a new cache pool catalog object type was introduced. The catalog server
caches the known pools (periodically refreshing the cache) and sends the known pools out
in catalog updates. This allows impalads to perform analysis checks on cache pool
existence going to HDFS. It would be easy to use this to add basic cache pool management
in the future (ADD/DROP/SHOW CACHE POOL).

Waiting for the table/partition to become cached may take a long time. Instead of
blocking the user from access the time during this period we will wait for the cache
requests to complete in the background and once they have finished the table metadata
will be automatically refreshed.

Change-Id: I1de9c6e25b2a3bdc09edebda5510206eda3dd89b
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2310
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2711
Tested-by: Lenni Kuff <lskuff@cloudera.com>
[CDH4] Disable HDFS caching in analysis on CDH4

Disables HDFS caching and caching tests in analysis on CDH4.

Change-Id: I4f193ca205cb4408bbdb63fc160dd89f39a7165f
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2712
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: Lenni Kuff <lskuff@cloudera.com>
Fix impala-config.sh when JAVA_HOME not set

Change-Id: Iaefda2039de1a5aafc782bca582d3007abcf6eff
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2803
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins