GitHub PushEvent at 2014-12-09 19:31:01
Repository: spark
Actor: jkbradley
Payload: {"shas":[["794f3aec24acb578e258532ad0590554d07958ba","f77d28eacd4f9a33b9347fca996a1e90fc43f180@gmail.com","[SPARK-4745] Fix get_existing_cluster() function with multiple security groups\n\nThe current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups == cluster_name + \"-master\" (or \"-slaves\"). This fix allows for multiple security groups by checking if the cluster_name + \"-master\" security group is in the list of groups for a particular instance.\n\nAuthor: alexdebrie <alexdebrie1@gmail.com>\n\nCloses #3596 from alexdebrie/master and squashes the following commits:\n\n9d51232 [alexdebrie] Fix get_existing_cluster() function with multiple security groups","alexdebrie",false],["743a889d2778f797aabc3b1e8146e7aa32b62a48","bc876e5da4db1e11ccdcfbe9c753b0d4123363b9@phusca-l24858.wlan.na.novartis.net","[SPARK-4459] Change groupBy type parameter from K to U\n\nPlease see https://issues.apache.org/jira/browse/SPARK-4459\n\nAuthor: Saldanha <saldaal1@phusca-l24858.wlan.na.novartis.net>\n\nCloses #3327 from alokito/master and squashes the following commits:\n\n54b1095 [Saldanha] [SPARK-4459] changed type parameter for keyBy from K to U\nd5f73c3 [Saldanha] [SPARK-4459] added keyBy test\n316ad77 [Saldanha] SPARK-4459 changed type parameter for groupBy from K to U.\n62ddd4b [Saldanha] SPARK-4459 added failing unit test","Saldanha",false],["ab8177da2defab1ecd8bc0cd5a21f07be5b8d2c5","bb02c6aa60d4a0a091a9679b63db9e2367e07b50@me.com","[SPARK-4652][DOCS] Add docs about spark-git-repo option\n\nThere might be some cases when WIPS spark version need to be run\non EC2 cluster. In order to setup this type of cluster more easily,\nadd --spark-git-repo option description to ec2 documentation.\n\nAuthor: lewuathe <lewuathe@me.com>\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #3513 from Lewuathe/doc-for-development-spark-cluster and squashes the following commits:\n\n6dae8ee [lewuathe] Wrap consistent with other descriptions\ncfaf9be [lewuathe] Add docs about spark-git-repo option\n\n(Editing / cleanup by Josh Rosen)","lewuathe",false],["ed92b47e83c2882f0e76da78dc268577df820382","c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","[SPARK-4397] Move object RDD to the front of RDD.scala.\n\nI ran into multiple cases that SBT/Scala compiler was confused by the implicits in continuous compilation mode. Adding explicit return types fixes the problem.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #3580 from rxin/rdd-implicit and squashes the following commits:\n\nee32fcd [Reynold Xin] Move object RDD to the end of the file.\nb8562c9 [Reynold Xin] Merge branch 'master' of github.com:apache/spark into rdd-implicit\nd4e9f85 [Reynold Xin] Code review.\na836a37 [Reynold Xin] Move object RDD to the front of RDD.scala.","Reynold Xin",false],["ddfc09c36381a0880dfa6778be2ca0bc7d80febf","e2aa89bc956dec4bf9c6f5e2da3f58db06fb878d@oss.nttdata.co.jp","[SPARK-4421] Wrong link in spark-standalone.html\n\nModified the link of building Spark.\n\nAuthor: Masayoshi TSUZUKI <tsudukim@oss.nttdata.co.jp>\n\nCloses #3279 from tsudukim/feature/SPARK-4421 and squashes the following commits:\n\n56e31c1 [Masayoshi TSUZUKI] Modified the link of building Spark.","Masayoshi TSUZUKI",false],["15cf3b0125fe238dea2ce13e703034ba7cef477f","9977d5aadd55c92e41457706313fdbf5d91867c6@gmail.com","Fix typo in Spark SQL docs.\n\nAuthor: Andy Konwinski <andykonwinski@gmail.com>\n\nCloses #3611 from andyk/patch-3 and squashes the following commits:\n\n7bab333 [Andy Konwinski] Fix typo in Spark SQL docs.","Andy Konwinski",false],["ca379039f701e423fa07933db4e063cb85d0236a","e2aa89bc956dec4bf9c6f5e2da3f58db06fb878d@oss.nttdata.co.jp","[SPARK-4464] Description about configuration options need to be modified in docs.\n\nAdded description about -h and -host.\nModified description about -i and -ip which are now deprecated.\nAdded description about --properties-file.\n\nAuthor: Masayoshi TSUZUKI <tsudukim@oss.nttdata.co.jp>\n\nCloses #3329 from tsudukim/feature/SPARK-4464 and squashes the following commits:\n\n6c07caf [Masayoshi TSUZUKI] [SPARK-4464] Description about configuration options need to be modified in docs.","Masayoshi TSUZUKI",false],["87437df036305c6f467dc6e0bf7891d27d266a6b","02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","Revert \"[HOT FIX] [YARN] Check whether `/lib` exists before listing its files\"\n\nThis reverts commit 90ec643e9af4c8bbb9000edca08c07afb17939c7.","Andrew Or",false],["fd8525334c443fbdc7dd58a62cb1aecae51b460b","02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","Revert \"SPARK-2624 add datanucleus jars to the container in yarn-cluster\"\n\nThis reverts commit a975dc32799bb8a14f9e1c76defaaa7cfbaf8b53.","Andrew Or",false],["f5801e813f3c2573ebaf1af839341489ddd3ec78","17b9e1c64588c7fa6419b4d29dc1f4426279ba01@databricks.com","[SPARK-4753][SQL] Use catalyst for partition pruning in newParquet.\n\nAuthor: Michael Armbrust <michael@databricks.com>\n\nCloses #3613 from marmbrus/parquetPartitionPruning and squashes the following commits:\n\n4f138f8 [Michael Armbrust] Use catalyst for partition pruning in newParquet.","Michael Armbrust",false],["6f61e1f961826a6c9e98a66d10b271b7e3c7dd55","e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","[SPARK-4761][SQL] Enables Kryo by default in Spark SQL Thrift server\n\nEnables Kryo and disables reference tracking by default in Spark SQL Thrift server. Configurations explicitly defined by users in `spark-defaults.conf` are respected (the Thrift server is started by `spark-submit`, which handles configuration properties properly).\n\n<!-- Reviewable:start -->\n[<img src=\"https://reviewable.io/review_button.png\" height=40 alt=\"Review on Reviewable\"/>](https://reviewable.io/reviews/apache/spark/3621)\n<!-- Reviewable:end -->\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #3621 from liancheng/kryo-by-default and squashes the following commits:\n\n70c2775 [Cheng Lian] Enables Kryo by default in Spark SQL Thrift server","Cheng Lian",false],["98a7d09978eeb775600ff41f9cc6ae8622026b71","7632d88ebdd73175d662531eefd9a84b8ade5e24@intel.com","[SPARK-4005][CORE] handle message replies in receive instead of in the individual private methods\n\nIn BlockManagermasterActor, when handling message type UpdateBlockInfo, the message replies is in handled in individual private methods, should handle it in receive of Akka.\n\nAuthor: Zhang, Liye <liye.zhang@intel.com>\n\nCloses #2853 from liyezhang556520/akkaRecv and squashes the following commits:\n\n9b06f0a [Zhang, Liye] remove the unreachable code\nbf518cd [Zhang, Liye] change the indent\n242166b [Zhang, Liye] modified accroding to the comments\nd4b929b [Zhang, Liye] [SPARK-4005][CORE] handle message replies in receive instead of in the individual private methods","Zhang, Liye",false],["6eb1b6f6204ea3c8083af3fb9cd990d9f3dac89d","fbb1b1249c4fec363b436c739458ff446a82c033@gmail.com","Streaming doc : do you mean inadvertently?\n\nAuthor: CrazyJvm <crazyjvm@gmail.com>\n\nCloses #3620 from CrazyJvm/streaming-foreachRDD and squashes the following commits:\n\nb72886b [CrazyJvm] do you mean inadvertently?","CrazyJvm",false],["e895e0cbecbbec1b412ff21321e57826d2d0a982","48d3a847ef684d87177a99cd6c21cdf466f2e98a@qq.com","[SPARK-3623][GraphX] GraphX should support the checkpoint operation\n\nAuthor: GuoQiang Li <witgo@qq.com>\n\nCloses #2631 from witgo/SPARK-3623 and squashes the following commits:\n\na70c500 [GuoQiang Li] Remove java related\n4d1e249 [GuoQiang Li] Add comments\ne682724 [GuoQiang Li] Graph should support the checkpoint operation","GuoQiang Li",false],["2e6b736b0e6e5920d0523533c87832a53211db42","e58867706a26c17cde71f0f9f5e010ffb994f9e8@gmail.com","[SPARK-4646] Replace Scala.util.Sorting.quickSort with Sorter(TimSort) in Spark\n\nThis patch just replaces a native quick sorter with Sorter(TimSort) in Spark.\nIt could get performance gains by ~8% in my quick experiments.\n\nAuthor: Takeshi Yamamuro <linguin.m.s@gmail.com>\n\nCloses #3507 from maropu/TimSortInEdgePartitionBuilderSpike and squashes the following commits:\n\n8d4e5d2 [Takeshi Yamamuro] Remove a wildcard import\n3527e00 [Takeshi Yamamuro] Replace Scala.util.Sorting.quickSort with Sorter(TimSort) in Spark","Takeshi Yamamuro",false],["8817fc7fe8785d7b11138ca744f22f7e70f1f0a0","e58867706a26c17cde71f0f9f5e010ffb994f9e8@gmail.com","[SPARK-4620] Add unpersist in Graph and GraphImpl\n\nAdd an IF to uncache both vertices and edges of Graph/GraphImpl.\nThis IF is useful when iterative graph operations build a new graph in each iteration, and the vertices and edges of previous iterations are no longer needed for following iterations.\n\nAuthor: Takeshi Yamamuro <linguin.m.s@gmail.com>\n\nThis patch had conflicts when merged, resolved by\nCommitter: Ankur Dave <ankurdave@gmail.com>\n\nCloses #3476 from maropu/UnpersistInGraphSpike and squashes the following commits:\n\n77a006a [Takeshi Yamamuro] Add unpersist in Graph and GraphImpl","Takeshi Yamamuro",false],["ab2abcb5ef925f15fa0e08d34a79b94a7b6578ef","cdc781d9ee41870657a6bfbe4384e9f2f1f2a5dd@kelkoo.com","[SPARK-4764] Ensure that files are fetched atomically\n\ntempFile is created in the same directory than targetFile, so that the\nmove from tempFile to targetFile is always atomic\n\nAuthor: Christophe Préaud <christophe.preaud@kelkoo.com>\n\nCloses #2855 from preaudc/master and squashes the following commits:\n\n9ba89ca [Christophe Préaud] Ensure that files are fetched atomically\n54419ae [Christophe Préaud] Merge remote-tracking branch 'upstream/master'\nc6a5590 [Christophe Préaud] Revert commit 8ea871f8130b2490f1bad7374a819bf56f0ccbbd\n7456a33 [Christophe Préaud] Merge remote-tracking branch 'upstream/master'\n8ea871f [Christophe Préaud] Ensure that files are fetched atomically","Christophe Préaud",false],["d6a972b3e4dc35a2d95df47d256462b325f4bda6","e1a8399e36effe75a124102eb2a2bdfc82e0aab4@cloudera.com","[SPARK-4774] [SQL] Makes HiveFromSpark more portable\n\nHiveFromSpark read the kv1.txt file from SPARK_HOME/examples/src/main/resources/kv1.txt which assumed\nyou had a source tree checked out. Now we copy the kv1.txt file to a temporary file and delete it when\nthe jvm shuts down. This allows us to run this example outside of a spark source tree.\n\nAuthor: Kostas Sakellis <kostas@cloudera.com>\n\nCloses #3628 from ksakellis/kostas-spark-4774 and squashes the following commits:\n\n6770f83 [Kostas Sakellis] [SPARK-4774] [SQL] Makes HiveFromSpark more portable","Kostas Sakellis",true],["65f929d5b3a50a73cd6397bd4b72c3e7d94c99d7","02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","[SPARK-4750] Dynamic allocation - synchronize kills\n\nSimple omission on my part.\n\nAuthor: Andrew Or <andrew@databricks.com>\n\nCloses #3612 from andrewor14/dynamic-allocation-synchronization and squashes the following commits:\n\n1f03b60 [Andrew Or] Synchronize kills","Andrew Or",true],["e829bfa1ab9b68f44c489d26efb042f793fd9362","291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","SPARK-3926 [CORE] Reopened: result of JavaRDD collectAsMap() is not serializable\n\nMy original 'fix' didn't fix at all. Now, there's a unit test to check whether it works. Of the two options to really fix it -- copy the `Map` to a `java.util.HashMap`, or copy and modify Scala's implementation in `Wrappers.MapWrapper`, I went with the latter.\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #3587 from srowen/SPARK-3926 and squashes the following commits:\n\n8586bb9 [Sean Owen] Remove unneeded no-arg constructor, and add additional note about copied code in LICENSE\n7bb0e66 [Sean Owen] Make SerializableMapWrapper actually serialize, and add unit test","Sean Owen",true]],"size":26,"ref":"refs/heads/master","head":"912563aa3553afc0871d5b5858f533aa39cb99e5"}