HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be
cast LazyInteger]
(Navis Ryu via Ashutosh Chauhan)

Summary:
DPAL-1111 GROUP BY causing ClassCastException [LazyDioInteger cannot be cast
LazyInteger]

This relates to https://issues.apache.org/jira/browse/HIVE-1634.

The following work fine:

CREATE EXTERNAL TABLE tim_hbase_occurrence (
  id int,
  scientific_name string,
  data_resource_id int
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES (
  "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b"
) TBLPROPERTIES(
  "hbase.table.name" = "mini_occurrences",
  "hbase.table.default.storage.type" = "binary"
);
SELECT * FROM tim_hbase_occurrence LIMIT 3;
SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;

However, the following fails:

SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY
data_resource_id;

The error given:

0 TS
2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initialization Done 7 MAP
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Processing alias tim_hbase_occurrence for file
hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7
forwarding 1 rows
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0
forwarding 1 rows
2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1
forwarding 1 rows
2012-04-17 16:58:45,723 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"id":1444,"scientific_name":null,"data_resource_id":1081}
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
	at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
	at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
	at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
	at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
	at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
	... 9 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
	at
org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
	at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
	at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
	at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142)
	at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119)
	at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750)
	at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
	... 18 more

Test Plan: EMPTY

Reviewers: JIRA, ashutoshc

Reviewed By: ashutoshc

Differential Revision: https://reviews.facebook.net/D2871

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1328157 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2965 : Revert HIVE-2612 (hashutosh)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1328469 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2966 :Revert HIVE-2795  (Thejas Nair via Ashutosh Chauhan)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1328568 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2970: several jars in hive tar generated are not required (Thejas Nair via Ashutosh Chauhan)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1329314 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2646. Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs (Andrew Bayer and Thomas Weise)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1329381 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2928. Support for Oracle-backed Hive-Metastore (longvarchar to clob in package.jdo) (Mithun Radhakrishnan and Andrew Bayer via cws)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1329416 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2883 [jira] Metastore client doesnt close connection properly

Summary:
https://issues.apache.org/jira/browse/HIVE-2883

Here is the patch which gets rid of this problem as per my hypothesis. I also
tested it and the above noted stack trace doesnt occur anymore.

While closing connection, it always fail with following trace. Seemingly, it
doesnt have any harmful effects.

12/03/20 10:55:02 ERROR hive.metastore: Unable to shutdown local metastore
client
org.apache.thrift.transport.TTransportException: Cannot write to null
outputStream
	at
org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:142)
	at
org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
	at
org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91)
	at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
	at
com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:421)
	at com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:415)
	at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:310)

Test Plan: EMPTY

Reviewers: JIRA, cwsteinbach

Reviewed By: cwsteinbach

Differential Revision: https://reviews.facebook.net/D2613

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1329461 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2703 ResultSetMetaData.getColumnType() always returns VARCHAR(string) for partition columns
irrespective of partition column type (tamtam180 via namit)



git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1329492 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2803 [jira] utc_from_timestamp and utc_to_timestamp returns incorrect
results.
(Kiyotaka Suzuki via Carl Steinbach)

Summary:
https://issues.apache.org/jira/browse/HIVE-2803

HIVE-2803 utc_from_timestamp and utc_to_timestamp returns incorrect results.

It changed so that applyOffset() return a new instance.

How to reproduce:


$ echo "2011-12-25 09:00:00.123456" > /tmp/data5.txt
hive> create table ts1(t1 timestamp);
hive> load data local inpath '/tmp/data5.txt' overwrite into table ts1;
hive> select t1, from_utc_timestamp(t1, 'JST'), from_utc_timestamp(t1, 'JST')
from ts1 limit 1;



The following result is expected:

 2011-12-25 09:00:00.123456      2011-12-25 18:00:00.123456      2011-12-25
18:00:00.123456


However, the above query return incorrect result like this:

 2011-12-26 03:00:00.492456      2011-12-26 03:00:00.492456      2011-12-26
03:00:00.492456



This is because GenericUDFFromUtcTimestamp.applyOffset() does setTime()
improperly.
On evaluating query, timestamp argument always returns the same instance.
GenericUDFFromUtcTimestamp.applyOffset() does setTime() on the instance.
That means it adds all offsets in the query.

Test Plan: EMPTY

Reviewers: JIRA

Differential Revision: https://reviews.facebook.net/D1959

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1329507 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2976. Fix maven-build Ant target (Thomas Weise and Andrew Bayer via cws)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1329997 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2960 Stop testing concat of partitions containing control characters.
(Kevin Wilfong via namit)



git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1330057 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2918. Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI. (cwsteinbach via kevinwilfong)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1330417 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2721 ability to select a view qualified by the database / schema name
(Martin Traverso via namit)



git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1330991 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2904 ant gen-test failed
(Carl Stienbach via namit)



git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1331162 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2986 Create the hooks (njain via kevinwilfong)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1332873 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2990 Remove hadoop-source Ivy resolvers and Ant targets (Carl Steinbach via Ashutosh Chauhan)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1333249 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2994 Pass a environment context to metastore thrift APIs
(Delia David via namit)



git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1334053 13f79535-47bb-0310-9956-ffa450edef68
HIVE-3002 Revert HIVE-2986
(Kevin Wilfong via namit)



git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1334060 13f79535-47bb-0310-9956-ffa450edef68
HIVE-2529 [jira] metastore 0.8 upgrade script for PostgreSQL
(Zhenxiao Luo via Carl Steinbach)

Summary:
HIVE-2529:  metastore 0.8 upgrade script for PostgreSQL

I think you mentioned that this was in the works.

Test Plan: EMPTY

Reviewers: JIRA, cwsteinbach

Reviewed By: cwsteinbach

Differential Revision: https://reviews.facebook.net/D3027

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1334537 13f79535-47bb-0310-9956-ffa450edef68
HIVE-3000 Potential infinite loop / log spew in ZookeeperHiveLockManager (njain via kevinwilfong)

git-svn-id: https://svn.apache.org/repos/asf/hive/trunk@1335106 13f79535-47bb-0310-9956-ffa450edef68