GitHub PushEvent at 2014-07-03 16:06:31
Repository: spark
Actor: tbrooks8
Payload: {"shas":[["b8f2e13aec715e038bd6d1d07b607683f138ac83","ba8a5fb55df29114fc9a1828df41a68b9ccf7e29@gmail.com","[SPARK-2233] make-distribution script should list the git hash in the RELEASE file\n\nThis patch adds the git revision hash (short version) to the RELEASE file. It uses git instead of simply checking for the existence of .git, so as to make sure that this is a functional repository.\n\nAuthor: Guillaume Ballet <gballet@gmail.com>\n\nCloses #1216 from gballet/master and squashes the following commits:\n\neabc50f [Guillaume Ballet] Refactored the script to take comments into account.\nd93e5e8 [Guillaume Ballet] [SPARK 2233] make-distribution script now lists the git hash tag in the RELEASE file.","Guillaume Ballet",true],["3c104c79d24425786cec0034f269ba19cf465b31","1fa2ef4755a9226cb9a0a4840bd89b158ac71391@redhat.com","[SPARK-1394] Remove SIGCHLD handler in worker subprocess\n\nIt should not be the responsibility of the worker subprocess, which\ndoes not intentionally fork, to try and cleanup child processes. Doing\nso is complex and interferes with operations such as\nplatform.system().\n\nIf it is desirable to have tighter control over subprocesses, then\nnamespaces should be used and it should be the manager's resposibility\nto handle cleanup.\n\nAuthor: Matthew Farrellee <matt@redhat.com>\n\nCloses #1247 from mattf/SPARK-1394 and squashes the following commits:\n\nc36f308 [Matthew Farrellee] [SPARK-1394] Remove SIGCHLD handler in worker subprocess","Matthew Farrellee",true],["2053d793cc2e8e5f5776e6576ddc6f8e6168e60c","c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@apache.org","Improve MapOutputTracker error logging.\n\nAuthor: Reynold Xin <rxin@apache.org>\n\nCloses #1258 from rxin/mapOutputTracker and squashes the following commits:\n\na7c95b6 [Reynold Xin] Improve MapOutputTracker error logging.","Reynold Xin",true],["cdf613fc52d7057555a2dbf2241ce90bacbabb4a","c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@apache.org","[SPARK-2320] Reduce exception/code block font size in web ui\n\nAuthor: Reynold Xin <rxin@apache.org>\n\nCloses #1261 from rxin/ui-pre-size and squashes the following commits:\n\n7ab1a69 [Reynold Xin] [SPARK-2320] Reduce exception/code block font size in web ui","Reynold Xin",true],["7b71a0e09622e09285a9884ebb67b5fb1c5caa53","284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","[SPARK-1683] Track task read metrics.\n\nThis commit adds a new metric in TaskMetrics to record\nthe input data size and displays this information in the UI.\n\nAn earlier version of this commit also added the read time,\nwhich can be useful for diagnosing straggler problems,\nbut unfortunately that change introduced a significant performance\nregression for jobs that don't do much computation. In order to\ntrack read time, we'll need to do sampling.\n\nThe screenshots below show the UI with the new \"Input\" field,\nwhich I added to the stage summary page, the executor summary page,\nand the per-stage page.\n\n![image](https://cloud.githubusercontent.com/assets/1108612/3167930/2627f92a-eb77-11e3-861c-98ea5bb7a1a2.png)\n\n![image](https://cloud.githubusercontent.com/assets/1108612/3167936/475a889c-eb77-11e3-9706-f11c48751f17.png)\n\n![image](https://cloud.githubusercontent.com/assets/1108612/3167948/80ebcf12-eb77-11e3-87ed-349fce6a770c.png)\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #962 from kayousterhout/read_metrics and squashes the following commits:\n\nf13b67d [Kay Ousterhout] Correctly format input bytes on executor page\n8b70cde [Kay Ousterhout] Added comment about potential inaccuracy of bytesRead\nd1016e8 [Kay Ousterhout] Udated SparkListenerSuite test\n8461492 [Kay Ousterhout] Miniscule style fix\nae04d99 [Kay Ousterhout] Remove input metrics for parallel collections\n719f19d [Kay Ousterhout] Style fixes\nbb6ec62 [Kay Ousterhout] Small fixes\n869ac7b [Kay Ousterhout] Updated Json tests\n44a0301 [Kay Ousterhout] Fixed accidentally added line\n4bd0568 [Kay Ousterhout] Added input source, renamed Hdfs to Hadoop.\nf27e535 [Kay Ousterhout] Updates based on review comments and to fix rebase\nbf41029 [Kay Ousterhout] Updated Json tests to pass\n0fc33e0 [Kay Ousterhout] Added explicit backward compatibility test\n4e52925 [Kay Ousterhout] Added Json output and associated tests.\n365400b [Kay Ousterhout] [SPARK-1683] Track task read metrics.","Kay Ousterhout",true],["66135a341d9f8baecc149d13ae5511f14578c395","79478dbbc8641d1d6545f348f655464aefbda99b@intel.com","[SPARK-2104] Fix task serializing issues when sort with Java non serializable class\n\nDetails can be see in [SPARK-2104](https://issues.apache.org/jira/browse/SPARK-2104). This work is based on Reynold's work, add some unit tests to validate the issue.\n\n@rxin , would you please take a look at this PR, thanks a lot.\n\nAuthor: jerryshao <saisai.shao@intel.com>\n\nCloses #1245 from jerryshao/SPARK-2104 and squashes the following commits:\n\nc8ee362 [jerryshao] Make field partitions transient\n2b41917 [jerryshao] Minor changes\n47d763c [jerryshao] Fix task serializing issue when sort with Java non serializable class","jerryshao",true],["a484030dae9d0d7e4b97cc6307e9e928c07490dc","f1f4455a46e181f5a91079e1a9678b51f3800f4d@redhat.com","SPARK-897:  preemptively serialize closures\n\nThese commits cause `ClosureCleaner.clean` to attempt to serialize the cleaned closure with the default closure serializer and throw a `SparkException` if doing so fails.  This behavior is enabled by default but can be disabled at individual callsites of `SparkContext.clean`.\n\nCommit 98e01ae8 fixes some no-op assertions in `GraphSuite` that this work exposed; I'm happy to put that in a separate PR if that would be more appropriate.\n\nAuthor: William Benton <willb@redhat.com>\n\nCloses #143 from willb/spark-897 and squashes the following commits:\n\nbceab8a [William Benton] Commented DStream corner cases for serializability checking.\n64d04d2 [William Benton] FailureSuite now checks both messages and causes.\n3b3f74a [William Benton] Stylistic and doc cleanups.\nb215dea [William Benton] Fixed spurious failures in ImplicitOrderingSuite\nbe1ecd6 [William Benton] Don't check serializability of DStream transforms.\nabe816b [William Benton] Make proactive serializability checking optional.\n5bfff24 [William Benton] Adds proactive closure-serializablilty checking\ned2ccf0 [William Benton] Test cases for SPARK-897.","William Benton",true],["6803642253b45c80ef8970606e350adabe4211ea","02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@andrewash.com","SPARK-2077 Log serializer that actually ends up being used\n\nI could settle with this being a debug also if we provided an example of how to turn it on in `log4j.properties`\n\nhttps://issues.apache.org/jira/browse/SPARK-2077\n\nAuthor: Andrew Ash <andrew@andrewash.com>\n\nCloses #1017 from ash211/SPARK-2077 and squashes the following commits:\n\n580f680 [Andrew Ash] Drop to debug\n0266415 [Andrew Ash] SPARK-2077 Log serializer that actually ends up being used","Andrew Ash",true],["358ae1534d01ad9e69364a21441a7ef23c2cb516","c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@apache.org","[SPARK-2322] Exception in resultHandler should NOT crash DAGScheduler and shutdown SparkContext.\n\nThis should go into 1.0.1.\n\nAuthor: Reynold Xin <rxin@apache.org>\n\nCloses #1264 from rxin/SPARK-2322 and squashes the following commits:\n\nc77c07f [Reynold Xin] Added comment to SparkDriverExecutionException and a test case for accumulator.\n5d8d920 [Reynold Xin] [SPARK-2322] Exception in resultHandler could crash DAGScheduler and shutdown SparkContext.","Reynold Xin",true],["5fccb567b37a94445512c7ec20b830b5e062089f","c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@apache.org","[SPARK-2318] When exiting on a signal, print the signal name first.\n\nAuthor: Reynold Xin <rxin@apache.org>\n\nCloses #1260 from rxin/signalhandler1 and squashes the following commits:\n\n8e73552 [Reynold Xin] Uh add Logging back in ApplicationMaster.\n0402ba8 [Reynold Xin] Synchronize SignalLogger.register.\ndc70705 [Reynold Xin] Added SignalLogger to YARN ApplicationMaster.\n79a21b4 [Reynold Xin] Added license header.\n0da052c [Reynold Xin] Added the SignalLogger itself.\ne587d2e [Reynold Xin] [SPARK-2318] When exiting on a signal, print the signal name first.","Reynold Xin",true],["04fa1223ee69760f0d23b40e56f4b036aa301879","291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","SPARK-2293. Replace RDD.zip usage by map with predict inside.\n\nThis is the only occurrence of this pattern in the examples that needs to be replaced. It only addresses the example change.\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #1250 from srowen/SPARK-2293 and squashes the following commits:\n\n6b1b28c [Sean Owen] Compute prediction-and-label RDD directly rather than by zipping, for efficiency","Sean Owen",true],["3319a3e3c604f187ff8176597b269af04ca5c1c5","f42d7ebcb1d93aaeb6711c7bde095ae44b1a1626@redhat.com","SPARK-2332 [build] add exclusion for old servlet-api on hadoop-client in core\n\nFix for class of test suite failures in jenkins\n\nAuthor: Peter MacKinnon <pmackinn@redhat.com>\n\nCloses #1271 from pdmack/master and squashes the following commits:\n\ncfe59fd [Peter MacKinnon] exclude servlet-api in hadoop-client for sbt\n6f39fec [Peter MacKinnon] add exclusion for old servlet-api on hadoop-client in core","Peter MacKinnon",true],["05c3d90e3527114d3abc08c7cc87f6efef96ebdc","284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","[SPARK-2185] Emit warning when task size exceeds a threshold.\n\nThis functionality was added in an earlier commit but shortly\nafter was removed due to a bad git merge (totally my fault).\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #1149 from kayousterhout/warning_bug and squashes the following commits:\n\n3f1bb00 [Kay Ousterhout] Fixed Json tests\n462a664 [Kay Ousterhout] Removed task set name from warning message\ne89b2f6 [Kay Ousterhout] Fixed Json tests.\n7af424c [Kay Ousterhout] Emit warning when task size exceeds a threshold.","Kay Ousterhout",true],["6596392da0fc0fee89e22adfca239a3477dfcbab","fd726277649d3b2e3ee5d332dca6c89739b68aa5@gmail.com","update the comments in SqlParser\n\nSqlParser has been case-insensitive after https://github.com/apache/spark/commit/dab5439a083b5f771d5d5b462d0d517fa8e9aaf2 was merged\n\nAuthor: CodingCat <zhunansjtu@gmail.com>\n\nCloses #1275 from CodingCat/master and squashes the following commits:\n\n17931cd [CodingCat] update the comments in SqlParser","CodingCat",true],["5c6ec94da1bacd8e65a43acb92b6721493484e7b","487cb95357ce6df18bdad737998cd3687ebdbd6f@tid.es","SPARK-2186: Spark SQL DSL support for simple aggregations such as SUM and AVG\n\n**Description** This patch enables using the `.select()` function in SchemaRDD with functions such as `Sum`, `Count` and other.\n**Testing** Unit tests added.\n\nAuthor: Ximo Guanter Gonzalbez <ximo@tid.es>\n\nCloses #1211 from edrevo/add-expression-support-in-select and squashes the following commits:\n\nfe4a1e1 [Ximo Guanter Gonzalbez] Extend SQL DSL to functions\ne1d344a [Ximo Guanter Gonzalbez] SPARK-2186: Spark SQL DSL support for simple aggregations such as SUM and AVG","Ximo Guanter Gonzalbez",true],["1e2c26c83dd2e807cf0031ceca8b338a1a57cac6","f070f8f0c50caeda5841963fc7b57e73e59dad56@happy-camper.st","[SPARK-2328] [SQL] Add execution of `SHOW TABLES` before `TestHive.reset()`.\n\n`PruningSuite` is executed first of Hive tests unfortunately, `TestHive.reset()` breaks the test environment.\nTo prevent this, we must run a query before calling reset the first time.\n\nAuthor: Takuya UESHIN <ueshin@happy-camper.st>\n\nCloses #1268 from ueshin/issues/SPARK-2328 and squashes the following commits:\n\n043ceac [Takuya UESHIN] Add execution of `SHOW TABLES` before `TestHive.reset()`.","Takuya UESHIN",true],["bc7041a42dfa84312492ea8cae6fdeaeac4f6d1c","f070f8f0c50caeda5841963fc7b57e73e59dad56@happy-camper.st","[SPARK-2287] [SQL] Make ScalaReflection be able to handle Generic case classes.\n\nAuthor: Takuya UESHIN <ueshin@happy-camper.st>\n\nCloses #1226 from ueshin/issues/SPARK-2287 and squashes the following commits:\n\n32ef7c3 [Takuya UESHIN] Add execution of `SHOW TABLES` before `TestHive.reset()`.\n541dc8d [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287\nfac5fae [Takuya UESHIN] Remove unnecessary method receiver.\nd306e60 [Takuya UESHIN] Merge branch 'master' into issues/SPARK-2287\n7de5706 [Takuya UESHIN] Make ScalaReflection be able to handle Generic case classes.","Takuya UESHIN",true],["0a146cf5a2a614ea05a4ab7910add69d0732c039","1af863b3594a334e782a0f5b6ba7a4c0e841c3cb@staples.com","Started to implement the simplifiable decision tree","Tim Brooks",true],["865cb29fe11e1f0e2e2cd4d27fd5565961da11c7","1af863b3594a334e782a0f5b6ba7a4c0e841c3cb@staples.com","Deleted old decision tree. Renamed top node to root node","Tim Brooks",true],["a93ed7029acf930462048bd6cfc2e903fbe3054e","1af863b3594a334e782a0f5b6ba7a4c0e841c3cb@staples.com","Renamed the tree to original name","Tim Brooks",true]],"size":21,"ref":"refs/heads/master","head":"b70efa532aa1e183722d30d368976b4e660de038"}