GitHub PushEvent at 2014-03-02 14:19:57
Repository: spark
Actor: YuryYudin
Payload: {"shas":[["46dff34458096e5330073ca58e0723da52aeddcd","5673041ad99da73cfefa170a7081c0daed5d97b6@cloudera.com","SPARK-1051. On YARN, executors don't doAs submitting user\n\nThis reopens https://github.com/apache/incubator-spark/pull/538 against the new repo\n\nAuthor: Sandy Ryza <sandy@cloudera.com>\n\nCloses #29 from sryza/sandy-spark-1051 and squashes the following commits:\n\n708ce49 [Sandy Ryza] SPARK-1051. doAs submitting user in YARN","Sandy Ryza",true],["4ba3f70a4e385368d0e826ba261f8eb60c25c896","a1a405b82800b492a7268cba6bc58270d2b527de@apache.org","SPARK-1151: Update dev merge script to use spark.git instead of incubator-spark\n\nAuthor: Thomas Graves <tgraves@apache.org>\n\nCloses #47 from tgravescs/fix_merge_script and squashes the following commits:\n\n8209ab1 [Thomas Graves] Update dev merge script to use spark.git instead of incubator-spark","Thomas Graves",true],["556c56689bbc32c6cec0d07b57bd3ec73ceb243e","284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","[SPARK-979] Randomize order of offers.\n\nThis commit randomizes the order of resource offers to avoid scheduling\nall tasks on the same small set of machines.\n\nThis is a much simpler solution to SPARK-979 than #7.\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #27 from kayousterhout/randomize and squashes the following commits:\n\n435d817 [Kay Ousterhout] [SPARK-979] Randomize order of offers.","Kay Ousterhout",true],["9aa095711858ce8670e51488f66a3d7c1a821c30","f1b5a91d4d6ad523f2610114591c007e75d15084@apache.org","[SPARK-1150] fix repo location in create script\n\nhttps://spark-project.atlassian.net/browse/SPARK-1150\n\nfix the repo location in create_release script\n\nAuthor: Mark Grover <mark@apache.org>\n\nCloses #48 from CodingCat/script_fixes and squashes the following commits:\n\n01f4bf7 [Mark Grover] Fixing some nitpicks\nd2244d4 [Mark Grover] SPARK-676: Abbreviation in SPARK_MEM but not in SPARK_WORKER_MEMORY","Mark Grover",true],["ec992e182231da7313d85d10b3d5fd5975c44c8b","8bcd50e7b1eea79f965dde19d326001805380997@gmail.com","Revert \"[SPARK-1150] fix repo location in create script\"\n\nThis reverts commit 9aa095711858ce8670e51488f66a3d7c1a821c30.","Patrick Wendell",true],["fe195ae113941766b3921b1e4ec222ed830b5b8f","fd726277649d3b2e3ee5d332dca6c89739b68aa5@gmail.com","[SPARK-1150] fix repo location in create script (re-open)\n\nreopen for https://spark-project.atlassian.net/browse/SPARK-1150\n\nAuthor: CodingCat <zhunansjtu@gmail.com>\n\nCloses #52 from CodingCat/script_fixes and squashes the following commits:\n\nfc05a71 [CodingCat] fix repo location in create script","CodingCat",true],["3a8b698e961ac05d9d53e2bbf0c2844fcb1010d1","fd726277649d3b2e3ee5d332dca6c89739b68aa5@gmail.com","[SPARK-1100] prevent Spark from overwriting directory silently\n\nThanks for Diana Carroll to report this issue (https://spark-project.atlassian.net/browse/SPARK-1100)\n\nthe current saveAsTextFile/SequenceFile will overwrite the output directory silently if the directory already exists, this behaviour is not desirable because\n\noverwriting the data silently is not user-friendly\n\nif the partition number of two writing operation changed, then the output directory will contain the results generated by two runnings\n\nMy fix includes:\n\nadd some new APIs with a flag for users to define whether he/she wants to overwrite the directory:\nif the flag is set to true, then the output directory is deleted first and then written into the new data to prevent the output directory contains results from multiple rounds of running;\n\nif the flag is set to false, Spark will throw an exception if the output directory already exists\n\nchanged JavaAPI part\n\ndefault behaviour is overwriting\n\nTwo questions\n\nshould we deprecate the old APIs without such a flag?\n\nI noticed that Spark Streaming also called these APIs, I thought we don't need to change the related part in streaming? @tdas\n\nAuthor: CodingCat <zhunansjtu@gmail.com>\n\nCloses #11 from CodingCat/SPARK-1100 and squashes the following commits:\n\n6a4e3a3 [CodingCat] code clean\nef2d43f [CodingCat] add new test cases and code clean\nac63136 [CodingCat] checkOutputSpecs not applicable to FSOutputFormat\nec490e8 [CodingCat] prevent Spark from overwriting directory silently and leaving dirty directory","CodingCat",true],["6fc76e49c19310ec0d6cdf4754271ad09d652576","24f70c800c13ae23cbad3ee3edbc742a6ecfb50e@alpinenow.com","Initialized the regVal for first iteration in SGD optimizer\n\nPorted from https://github.com/apache/incubator-spark/pull/633\n\nIn runMiniBatchSGD, the regVal (for 1st iter) should be initialized\nas sum of sqrt of weights if it's L2 update; for L1 update, the same logic is followed.\n\nIt maybe not be important here for SGD since the updater doesn't take the loss\nas parameter to find the new weights. But it will give us the correct history of loss.\nHowever, for LBFGS optimizer we implemented, the correct loss with regVal is crucial to\nfind the new weights.\n\nAuthor: DB Tsai <dbtsai@alpinenow.com>\n\nCloses #40 from dbtsai/dbtsai-smallRegValFix and squashes the following commits:\n\n77d47da [DB Tsai] In runMiniBatchSGD, the regVal (for 1st iter) should be initialized as sum of sqrt of weights if it's L2 update; for L1 update, the same logic is followed.","DB Tsai",true],["012bd5fbc97dc40bb61e0e2b9cc97ed0083f37f6","17b9e1c64588c7fa6419b4d29dc1f4426279ba01@databricks.com","Merge the old sbt-launch-lib.bash with the new sbt-launcher jar downloading logic.\n\nThis allows developers to pass options (such as -D) to sbt.  I also modified the SparkBuild to ensure spark specific properties are propagated to forked test JVMs.\n\nAuthor: Michael Armbrust <michael@databricks.com>\n\nCloses #14 from marmbrus/sbtScripts and squashes the following commits:\n\nc008b18 [Michael Armbrust] Merge the old sbt-launch-lib.bash with the new sbt-launcher jar downloading logic.","Michael Armbrust",true],["b70823c91d4dff9be81badf6567e25aa1df4c574","e246a34b1b06d1c2bff4d4b32fc05feb94645a79@gmail.com","Update io.netty from 4.0.13 Final to 4.0.17.Final\n\nThis update contains a lot of bug fixes and some new perf improvements.\nIt is also binary compatible with the current 4.0.13.Final\n\nFor more information: http://netty.io/news/2014/02/25/4-0-17-Final.html\n\nAuthor: Binh Nguyen <ngbinh@gmail.com>\n\nAuthor: Binh Nguyen <ngbinh@gmail.com>\n\nCloses #41 from ngbinh/master and squashes the following commits:\n\na9498f4 [Binh Nguyen] update io.netty to 4.0.17.Final","Binh Nguyen",true],["1fd2bfd3dd6c27a54880f0d7a658b39f358aa804","8bcd50e7b1eea79f965dde19d326001805380997@gmail.com","Remove remaining references to incubation\n\nThis removes some loose ends not caught by the other (incubating -> tlp) patches. @markhamstra this updates the version as you mentioned earlier.\n\nAuthor: Patrick Wendell <pwendell@gmail.com>\n\nCloses #51 from pwendell/tlp and squashes the following commits:\n\nd553b1b [Patrick Wendell] Remove remaining references to incubation","Patrick Wendell",true],["46bcb9551eb918ac4a31cd4cca924b432f6dc352","5a7a6a189b47cd31d24667a751e4003cd04eced5@databricks.com","SPARK-1137: Make ZK PersistenceEngine not crash for wrong serialVersionUID\n\nPreviously, ZooKeeperPersistenceEngine would crash the whole Master process if\nthere was stored data from a prior Spark version. Now, we just delete these files.\n\nAuthor: Aaron Davidson <aaron@databricks.com>\n\nCloses #4 from aarondav/zookeeper2 and squashes the following commits:\n\nfa8b40f [Aaron Davidson] SPARK-1137: Make ZK PersistenceEngine not crash for wrong serialVersionUID","Aaron Davidson",true]],"size":12,"ref":"refs/heads/wdd-master","head":"46bcb9551eb918ac4a31cd4cca924b432f6dc352"}