Project: springxd
Issue XD-3710: Facing issue while running Spring XD batch job on HDP version 2.3.2.0-2950
***Version	Spring XD Version : spring-xd-1.3.0.RELEASE, spring-xd-1.3.0.RELEASE-yarn	OS & Version: Linux 2.6.32-431.29.2.el6.x86_64	Java Version: java version "1.7.0_65" ***Description	The simple word count map reduce job using spring xd is failing with inline error message. ***Steps to recreate the problem	1. Created a jar for simple word count map reduce job.	2. Created jar using information given in ( http://docs.spring.io/spring-hadoop/docs/2.0.2.RELEASE/reference/html/hadoop.html#hadoop:tasklet )	3. Once the final jar was ready, uploaded using "module upload --name test_mr_module --type job --file /home/user/jar/samplemrjob.jar"	4. After that created and deployed job using "job create --name test_mr_job --definition test_mr_module --deploy"	5. Finally launched using "job launch test_mr_job" which failed with inline error. ***Describe XD Deployment : Distributed Deployment Type : Distributed - YARN ( on AWS EC2 cloud ) Number of xd-admin’s and xd-container’s : 1 Admin and 3 Containers ***Describe Other Components Transport: Redis 3.0.1 ZooKeeper: Version 3.4.6.2.3.2.0-2950 Hadoop deployment Data Platform : Hortonworks HDP 2.3.2.0-2950 RDBMS: MySQL ***Error Message: ***************************************************** 05:29:52,673 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying job 'test_mr_job' 05:29:53,655 INFO DeploymentsPathChildrenCache-0 container.DeploymentListener - Deploying module [ModuleDescriptor@6e5af900 moduleName = 'test_mr_module', moduleLabel = 'test_mr_module', group = 'test_mr_job', sourceChannelName = [null], sinkChannelName = [null], index = 0, type = job, parameters = map[[empty]], children = list[[empty]]] 05:30:24,351 ERROR inbound.job:test_mr_job-redis:queue-inbound-channel-adapter1 step.AbstractStep - Encountered an error executing step teststep in job test_mr_job java.lang.IllegalArgumentException: Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:443)	.	.	. at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint.access$300(RedisQueueMessageDrivenEndpoint.java:54) at org.springframework.integration.redis.inbound.RedisQueueMessageDrivenEndpoint$ListenerTask.run(RedisQueueMessageDrivenEndpoint.java:323) at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.URISyntaxException: Illegal character in path at index 11: /hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework at java.net.URI$Parser.fail(URI.java:2848) at java.net.URI$Parser.checkChars(URI.java:3021) at java.net.URI$Parser.parseHierarchical(URI.java:3105) at java.net.URI$Parser.parse(URI.java:3063) at java.net.URI.(URI.java:588) at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:441) *****************************************************
Story points: 8