Project: springxd
Issue XD-3006: db.password not being passed when Sqoop job --create called from Spring-XD Batch job
Created a single definition Sqoop Job --create statement. When processed through Sqoop direct (see attach image for props in repos. jobname sqoop2) Notice: job definition set db.require.password = false and has a propname called db.password that holds the value for password. When same definition is created as a batch job via Spring-XD (see image jobname lookupd_job), the db.require.password prop is set to "TRUE" and the db.password is not even created as a row. This causes a password prompt when trying to execute the job in Sqoop direct, that was created via a spring-XD batch job. Note the job definition for Spring-XD job contains meta-connect with username and password defined as well as import jdbc: --connect username and password. When created via Sqoop (no Spring-XD) password is stored and job is able to execute with no prompt. Please see attachment for repository job values. Highlighted in yellow is the discrepant property and value attribution. Please reach out if there are questions or need further details. To automate Spring-XD batch job using Sqoop via Spring we can not be prompted for password. Spring-XD Job definition: job create sqoop_lookup_d_job_v1 --definition "sqoop --command=job --args='--create lookupd_job --meta-connect \"jdbc:postgresql://xx.xxx.xx.xx:5433/bdl?user=xxxxx&password=xxxxxx\" -- import --connect jdbc:oracle:thin:@xxxxxx.com:xxxx/GDWP --username xxxxx --password xxxxx --query \"SELECT ROW_WID, LKUP_ID, TRIM(TRANSLATE(LKUP_CD, CHR(10)||CHR(13), ''-'')) AS LKUP_CD, TRIM(TRANSLATE(LKUP_DESC, CHR(10)||CHR(13), ''-'')) AS LKUP_DESC, TRIM(TRANSLATE(LKUP_TYPE, CHR(10)||CHR(13), ''-'')) AS LKUP_TYPE, TRIM(TRANSLATE(LKUP_VAL, CHR(10)||CHR(13), ''-'')) AS LKUP_VAL, SRC_STRT_DT, SRC_END_DT, W_INSERT_DT, W_UPDATE_DT FROM MASTERDATA.W_LOOKUP_D WHERE 1=1 AND $CONDITIONS\" --incremental append --check-column ROW_WID --last-value 10150 --target-dir /user/monasj1/bdl/data/sqoop/jobtest --num-mappers 1 --fields-terminated-by \"|\" --append'" Sqoop Direct Definition: sqoop job --create sqoop2 --meta-connect 'jdbc:postgresql://xx.xxx.xx.xx:5433/bdl?user=xxxxx&password=xxxxxx' -- import --connect jdbc:oracle:thin:@xxxxxx.com:xxxx/GDWP --username xxxxx --password xxxxx --query "SELECT ROW_WID, LKUP_ID, TRIM(TRANSLATE(LKUP_CD, CHR(10)||CHR(13), '-')) AS LKUP_CD, TRIM(TRANSLATE(LKUP_DESC, CHR(10)||CHR(13), '-')) AS LKUP_DESC, TRIM(TRANSLATE(LKUP_TYPE, CHR(10)||CHR(13), '-')) AS LKUP_TYPE, TRIM(TRANSLATE(LKUP_VAL, CHR(10)||CHR(13), '-')) AS LKUP_VAL, SRC_STRT_DT, SRC_END_DT, W_INSERT_DT, W_UPDATE_DT FROM MASTERDATA.W_LOOKUP_D WHERE 1=1 AND \$CONDITIONS " --incremental append --check-column ROW_WID --last-value 10150 --target-dir /user/monasj1/bdl/data/sqoop/jobtest --num-mappers 1 --fields-terminated-by '|' --append
Story points: 5