Project: springxd
Issue XD-1322: Add way to provide module config options for XD on YARN
There seems to be some intersection with the work for this issue and the rationalization of how module properties are handled. There will be changes to configuration/property management support such that each module (source, sink, etc) will be able to also be overridden in spring-xd.yml (or wherever -Dspring.config.location points to. The HDFS sink module for example, will have default values based on it's OptionsMetadata and will be of the form ..
That means in the configuration for hdfs.xml sink, there would be a config section such as {code:xml} fs.default.name=${sink.hdfs.hd.fs} mapred.job.tracker=${sink.hdfs.hd.jt} yarn.resourcemanager.address=${sink.hdfs.hd.rm} mapreduce.framework.name=${sink.hdfs.mr.fw} {code} With default values defined by a HdfsSinkOptionsMetadata class. The hdfs.xml module file would not contain any references to a properties file. A file specified by -Dspring.config.location could override the values in a config section such as sink: hdfs: hd.fs : hdfs://foobarhost:8020 hd.jt : 10.123.123.123:9000 etc.
Story points: 5