GitHub PushEvent at 2014-07-09 19:20:01
Repository: spark
Actor: willb
Payload: {"shas":[["cc3e0a14daf756ff5c2d4e7916438e175046e5bb","17b9e1c64588c7fa6419b4d29dc1f4426279ba01@databricks.com","[SPARK-2395][SQL] Optimize common LIKE patterns.\n\nAuthor: Michael Armbrust <michael@databricks.com>\n\nCloses #1325 from marmbrus/slowLike and squashes the following commits:\n\n023c3eb [Michael Armbrust] add comment.\n8b421c2 [Michael Armbrust] Handle the case where the final % is actually escaped.\nd34d37e [Michael Armbrust] add periods.\n3bbf35f [Michael Armbrust] Roll back changes to SparkBuild\n53894b1 [Michael Armbrust] Fix grammar.\n4094462 [Michael Armbrust] Fix grammar.\n6d3d0a0 [Michael Armbrust] Optimize common LIKE patterns.","Michael Armbrust",true],["c8a2313cdf825e0191680a423d17619b5504ff89","9d468ac3e51a0dd55cd858437e2237db1fbaae1b@gmail.com","[SPARK-2403] Catch all errors during serialization in DAGScheduler\n\nhttps://issues.apache.org/jira/browse/SPARK-2403\n\nSpark hangs for us whenever we forget to register a class with Kryo. This should be a simple fix for that. But let me know if you have a better suggestion.\n\nI did not write a new test for this. It would be pretty complicated and I'm not sure it's worthwhile for such a simple change. Let me know if you disagree.\n\nAuthor: Daniel Darabos <darabos.daniel@gmail.com>\n\nCloses #1329 from darabos/spark-2403 and squashes the following commits:\n\n3aceaad [Daniel Darabos] Print full stack trace for miscellaneous exceptions during serialization.\n52c22ba [Daniel Darabos] Only catch NonFatal exceptions.\n361e962 [Daniel Darabos] Catch all errors during serialization in DAGScheduler.","Daniel Darabos",true],["b520b6453ed76926108e0bdd56114d16e1d86850","fbb1b1249c4fec363b436c739458ff446a82c033@gmail.com","SPARK-2400 : fix spark.yarn.max.executor.failures explaination\n\nAccording to\n```scala\n  private val maxNumExecutorFailures = sparkConf.getInt(\"spark.yarn.max.executor.failures\",\n    sparkConf.getInt(\"spark.yarn.max.worker.failures\", math.max(args.numExecutors * 2, 3)))\n```\ndefault value should be numExecutors * 2, with minimum of 3,  and it's same to the config\n`spark.yarn.max.worker.failures`\n\nAuthor: CrazyJvm <crazyjvm@gmail.com>\n\nCloses #1282 from CrazyJvm/yarn-doc and squashes the following commits:\n\n1a5f25b [CrazyJvm] remove deprecated config\nc438aec [CrazyJvm] fix style\n86effa6 [CrazyJvm] change expression\n211f130 [CrazyJvm] fix html tag\n2900d23 [CrazyJvm] fix style\na4b2e27 [CrazyJvm] fix configuration spark.yarn.max.executor.failures","CrazyJvm",true],["32516f866a32d51bfaa04685ae77ba216b4202d9","c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@apache.org","[SPARK-2409] Make SQLConf thread safe.\n\nAuthor: Reynold Xin <rxin@apache.org>\n\nCloses #1334 from rxin/sqlConfThreadSafetuy and squashes the following commits:\n\nc1e0a5a [Reynold Xin] Fixed the duplicate comment.\n7614372 [Reynold Xin] [SPARK-2409] Make SQLConf thread safe.","Reynold Xin",true],["e6f7bfcfbf6aff7a9f8cd8e0a2166d0bf62b0912","1d6e1cf70ec6f9ab28d3ea4b27a49a77654d370e@nizzoli.net","[SPARK-2362] Fix for newFilesOnly logic in file DStream\n\nThe newFilesOnly logic should be inverted: the logic should be that if the flag newFilesOnly==true then only start reading files older than current time. As the code is now if newFilesOnly==true then it will start to read files that are older than 0L (that is: every file in the directory).\n\nAuthor: Gabriele Nizzoli <mail@nizzoli.net>\n\nCloses #1077 from gabrielenizzoli/master and squashes the following commits:\n\n4f1d261 [Gabriele Nizzoli] Fix for newFilesOnly logic in file DStream","Gabriele Nizzoli",true],["bf04a390e40d60aa7fcc551501d25f3f9d38377c","a5c478d606458f6c868a5126471748acd4730a6f@gmail.com","[SPARK-2392] Executors should not start their own HTTP servers\n\nExecutors currently start their own unused HTTP file servers. This is because we use the same SparkEnv class for both executors and drivers, and we do not distinguish this case.\n\nIn the longer term, we should separate out SparkEnv for the driver and SparkEnv for the executors.\n\nAuthor: Andrew Or <andrewor14@gmail.com>\n\nCloses #1335 from andrewor14/executor-http-server and squashes the following commits:\n\n46ef263 [Andrew Or] Start HTTP server only on the driver","Andrew Or",true],["ac9cdc116e1c5fcb291a4ff168cac002a8058f05","24f70c800c13ae23cbad3ee3edbc742a6ecfb50e@alpinenow.com","[SPARK-2413] Upgrade junit_xml_listener to 0.5.1\n\nwhich fixes the following issues\n\n1) fix the class name to be fully qualified classpath\n2) make sure the the reporting time is in second not in miliseond, which causing JUnit HTML to report incorrect number\n3) make sure the duration of the tests are accumulative.\n\nAuthor: DB Tsai <dbtsai@alpinenow.com>\n\nCloses #1333 from dbtsai/dbtsai-junit and squashes the following commits:\n\nbbeac4b [DB Tsai] Upgrade junit_xml_listener to 0.5.1 which fixes the following issues","DB Tsai",true],["1114207cc8e4ef94cb97bbd5a2ef3ae4d51f73fa","b11211e2ab75b5698a272c87c5dceec1e47542af@gmail.com","[SPARK-2152][MLlib] fix bin offset in DecisionTree node aggregations (also resolves SPARK-2160)\n\nHi, this pull fixes (what I believe to be) a bug in DecisionTree.scala.\n\nIn the extractLeftRightNodeAggregates function, the first set of rightNodeAgg values for Regression are set in line 792 as follows:\n\nrightNodeAgg(featureIndex)(2 * (numBins - 2))\n  = binData(shift + (2 * numBins - 1)))\n\nThen there is a loop that sets the rest of the values, as in line 809:\n\nrightNodeAgg(featureIndex)(2 * (numBins - 2 - splitIndex)) =\n  binData(shift + (2 *(numBins - 2 - splitIndex))) +\n  rightNodeAgg(featureIndex)(2 * (numBins - 1 - splitIndex))\n\nBut since splitIndex starts at 1, this ends up skipping a set of binData values.\n\nThe changes here address this issue, for both the Regression and Classification cases.\n\nAuthor: johnnywalleye <jsondag@gmail.com>\n\nCloses #1316 from johnnywalleye/master and squashes the following commits:\n\n73809da [johnnywalleye] fix bin offset in DecisionTree node aggregations","johnnywalleye",true],["339441f545ca96de4a7d6271e32f16803539a8c8","284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","[SPARK-2384] Add tooltips to UI.\n\nThis patch adds tooltips to clarify some points of confusion in the UI.  When users mouse over some of the table headers (shuffle read, write, and input size) as well as over the \"scheduler delay\" metric shown for each stage, a black tool tip (see image below) pops up describing the metric in more detail.  After the tooltip mechanism is added by this commit, I imagine others may want to add more tooltips for other things in the UI, but I think this is a good starting point.\n\n![tooltip](https://cloud.githubusercontent.com/assets/1108612/3491905/994e179e-059f-11e4-92f2-c6c12d248d81.jpg)\n\nThis looks scary-big but much of it is adding the bootstrap tool tip JavaScript.\n\nAlso I have no idea what to put for the license in tooltip (I left it the same -- the Twitter apache header) or for JQuery (left it as nothing) -- @mateiz what's the right thing here?\n\ncc @pwendell @andrewor14 @rxin\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #1314 from kayousterhout/tooltips and squashes the following commits:\n\n19981b5 [Kay Ousterhout] Exclude non-licensed javascript files from style check\nd9ab5a9 [Kay Ousterhout] Response to Andrew's review\n7752449 [Kay Ousterhout] [SPARK-2384] Add tooltips to UI.","Kay Ousterhout",true],["0eb11527d13083ced215e3fda44ed849198a57cb","97dedfb2b2352f366f8c0196cec810b797d6040e@gmail.com","[STREAMING] SPARK-2343: Fix QueueInputDStream with oneAtATime false\n\nFix QueueInputDStream which was not removing dequeued items when used with the oneAtATime flag disabled.\n\nAuthor: Manuel Laflamme <manuel.laflamme@gmail.com>\n\nCloses #1285 from mlaflamm/spark-2343 and squashes the following commits:\n\n61c9e38 [Manuel Laflamme] Unit tests for queue input stream\nc51d029 [Manuel Laflamme] Fix QueueInputDStream with oneAtATime false","Manuel Laflamme",true],["d35e3db2325931492b64890125a70579bc3b587b","b11211e2ab75b5698a272c87c5dceec1e47542af@gmail.com","[SPARK-2417][MLlib] Fix DecisionTree tests\n\nFixes test failures introduced by https://github.com/apache/spark/pull/1316.\n\nFor both the regression and classification cases,\nval stats is the InformationGainStats for the best tree split.\nstats.predict is the predicted value for the data, before the split is made.\nSince 600 of the 1,000 values generated by DecisionTreeSuite.generateCategoricalDataPoints() are 1.0 and the rest 0.0, the regression tree and classification tree both correctly predict a value of 0.6 for this data now, and the assertions have been changed to reflect that.\n\nAuthor: johnnywalleye <jsondag@gmail.com>\n\nCloses #1343 from johnnywalleye/decision-tree-tests and squashes the following commits:\n\nef80603 [johnnywalleye] [SPARK-2417][MLlib] Fix DecisionTree tests","johnnywalleye",true],["1f33e1f2013c508aa86511750f7bd8437154e51a","b769df1c77761164afe3e6ca49fe2e5cd742e1da@twitter.com","SPARK-1782: svd for sparse matrix using ARPACK\n\ncopy ARPACK dsaupd/dseupd code from latest breeze\nchange RowMatrix to use sparse SVD\nchange tests for sparse SVD\n\nAll tests passed. I will run it against some large matrices.\n\nAuthor: Li Pu <lpu@twitter.com>\nAuthor: Xiangrui Meng <meng@databricks.com>\nAuthor: Li Pu <li.pu@outlook.com>\n\nCloses #964 from vrilleup/master and squashes the following commits:\n\n7312ec1 [Li Pu] very minor comment fix\n4c618e9 [Li Pu] Merge pull request #1 from mengxr/vrilleup-master\na461082 [Xiangrui Meng] make superscript show up correctly in doc\n861ec48 [Xiangrui Meng] simplify axpy\n62969fa [Xiangrui Meng] use BDV directly in symmetricEigs change the computation mode to local-svd, local-eigs, and dist-eigs update tests and docs\nc273771 [Li Pu] automatically determine SVD compute mode and parameters\n7148426 [Li Pu] improve RowMatrix multiply\n5543cce [Li Pu] improve svd api\n819824b [Li Pu] add flag for dense svd or sparse svd\neb15100 [Li Pu] fix binary compatibility\n4c7aec3 [Li Pu] improve comments\ne7850ed [Li Pu] use aggregate and axpy\n827411b [Li Pu] fix EOF new line\n9c80515 [Li Pu] use non-sparse implementation when k = n\nfe983b0 [Li Pu] improve scala style\n96d2ecb [Li Pu] improve eigenvalue sorting\ne1db950 [Li Pu] SPARK-1782: svd for sparse matrix using ARPACK","Li Pu",true]],"size":12,"ref":"refs/heads/master","head":"1f33e1f2013c508aa86511750f7bd8437154e51a"}