databricks/spark-knowledgebase — pull request #7 by JoshRosen (2014-10-08 20:39:30+00:00)
Title: Add "network connectivity issues" section

This PR adds a section on troubleshooting network connectivity issues between Spark components, such as SparkContext and the standalone Master and Workers. This section contains some very low-level details about the precedence between different configuration options that affect certain components' bound hostnames. It's also incomplete, since it doesn't cover components like the web UIs, but I think it's a decent start.

It would be great to expand the list of symptoms to include other types of connectivity issues, such as problems with workers connecting to standalone masters, or connectivity issues between SparkContext and executors. First, though, we need to get samples of logs to see how those errors manifest themselves.