spring-projects/spring-xd — pull request #591 by trisberg (2014-02-19 21:47:25+00:00)
Title: XD-1321 Creating shell scripts and support classes to start XD on YARN

- Adding xd-admin and xd-container scripts for YARN

- Adding YARN Client and Appmaster implementations

- Removing directory paths from xd-yarn zip distribution

To try this out:

1) install Hadoop 2.2.0 and start up dfs, yarn and historyserver
2) build xd distribution - build/distributions/ should now have a spring-xd-1.0.0.BUILD-SNAPSHOT-yarn.zip
3) unzip spring-xd-1.0.0.BUILD-SNAPSHOT-yarn.zip somewhere
4) the unzipped dir (spring-xd-1.0.0.BUILD-SNAPSHOT-yarn) contains a config/xd-site.yml file with localized settings
change anything that needs to be modified (should work with defaults if hadoop runs on localhost)
5) from the unzipped dir (spring-xd-1.0.0.BUILD-SNAPSHOT-yarn) run:
./bin/xd-admin.sh
./bin/xd-container.sh
6) check that there are two apps running in YARN:
yarn application -list

To get two apps running in YARN I had to modify my etc/hadoop/yarn-site.xml - added this:

yarn.nodemanager.resource.cpu-vcores
8

yarn.nodemanager.resource.memory-mb
8192

yarn.scheduler.minimum-allocation-mb
512

yarn.nodemanager.vmem-pmem-ratio
8.1

yarn.nodemanager.pmem-check-enabled
false

yarn.nodemanager.vmem-check-enabled
false

What you need would probably be a bit different, but this is a start.