Huawei-Spark/spark — pull request #35 by jackylk (2014-11-30 04:16:51+00:00)
Title: add serialVersionUID in HBaseRelation to solve serialization compatibility problem

Currently we are writing HBaseRelation object into the 'metadata' htable in catalog, and we are modifying HBaseRelation for new features which will generate a new serialVersionUID for every new version of it, so I am getting exception sometimes like this:

java.io.InvalidClassException: org.apache.spark.sql.hbase.HBaseRelation; local class incompatible: stream classdesc serialVersionUID = -6079691767473621545, local class serialVersionUID = 7749798971398340650
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at org.apache.spark.sql.hbase.HBaseCatalog.getRelationFromResult(HBaseCatalog.scala:334)
at org.apache.spark.sql.hbase.HBaseCatalog.getAllTableName(HBaseCatalog.scala:344)
at org.apache.spark.sql.hbase.execution.ShowTablesCommand.sideEffectResult$lzycompute(hbaseCommands.scala:114)
at org.apache.spark.sql.hbase.execution.ShowTablesCommand.sideEffectResult(hbaseCommands.scala:112)
at org.apache.spark.sql.execution.Command$class.execute(commands.scala:44)
at org.apache.spark.sql.hbase.execution.ShowTablesCommand.execute(hbaseCommands.scala:109)
at org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:390)
at org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:390)
at org.apache.spark.sql.SchemaRDDLike$class.$init$(SchemaRDDLike.scala:58)
at org.apache.spark.sql.SchemaRDD.(SchemaRDD.scala:104)
at org.apache.spark.sql.hbase.HBaseSQLContext.sql(HBaseSQLContext.scala:87)
at org.apache.spark.sql.hbase.HBaseSQLCLIDriver$.process(HBaseSQLCliDriver.scala:135)
at org.apache.spark.sql.hbase.HBaseSQLCLIDriver$.processLine(HBaseSQLCliDriver.scala:114)
at org.apache.spark.sql.hbase.HBaseSQLCLIDriver$.main(HBaseSQLCliDriver.scala:91)
at org.apache.spark.sql.hbase.HBaseSQLCLIDriver.main(HBaseSQLCliDriver.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:335)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

So the solution is to add serialVersionUID in HBaseRelation to solve this problem. The serialVersionUID I picked randomly, and I am not sure whether it is the best solution.
You can refer to http://stackoverflow.com/questions/285793/what-is-a-serialversionuid-and-why-should-i-use-it

And I think it is also better to move the build functions from HBaseRelation to another class, thus we can keep modification of HBaseRelation minimal. I will post another PR for this.