opened
ScrapCodes
username_0
This is work in progress, because I wanted some early feedback as to overall approach taken. At the moment with current code, this mounting feature does not work in client mode of submission. The reason is, in client mode kubernetes does not create any resources. 

My question is, what is the correct way to create a config map, while in the client mode. If it is created in a place like ExecutorPodAllocator, then since it is run repeatedly, then we would not want the config map to be created again and again on each executor pod allocation. So, what would be correct place to do it.

### What changes were proposed in this pull request?
A new feature to propagate correct logging configuration file either by picking up from user specified logger configuration file on the classpath or through user specified k8s config map.

### Why are the changes needed?
Please see referenced JIRA.

### Does this PR introduce any user-facing change?
No

### How was this patch tested?
Manually tested.
[WIP][SPARK-25065][k8s] Allow setting up correct logging configuration on driver and executor.
issue
created
ScrapCodes
username_0
@srowen and @vanzin Do you think this patch makes sense? I am ready to work on it more if some more work is needed. Thanks :)
comment
created
ScrapCodes
username_0
Jenkins, retest this please.
comment
created
ScrapCodes
username_0
Jenkins, retest this please.
comment
created
ScrapCodes
username_0
This failed with 20/02/04 00:20:45 INFO AmazonHttpClient: Unable to execute HTTP request: Read timed out
20/02/04 00:20:45.716 ScalaTest-main-running-KubernetesSuite INFO ProcessUtils: java.net.SocketTimeoutException: Read timed out
comment
created
ScrapCodes
username_0
This test is failing more often and only occasionally it passes.

- Launcher client dependencies *** FAILED ***
  
The code passed to eventually never returned normally. Attempted 1 times over 3.375042707433333 minutes. Last failure message: Failed to create bucket spark.. (DepsTestsSuite.scala:213)
comment
created
ScrapCodes
username_0
Jenkins, retest this please.
comment
created
ScrapCodes
username_0
@mccheah @vanzin @dongjoon-hyun Can you please take a look?
comment
created
ScrapCodes
username_0
@vanzin, Hi Marcelo Vanzin, I have added you for the review, as you have worked on similar features as this PR, please take a look.
comment