Source:
<code>public void run() throws Exception { String cdnURL = pickCDN(); if (!cdnURL.isEmpty()) { info("####################################"); info(" Distcp"); <START> info("####################################"); <END> info("Using CDN: " + cdnURL); try { String username = props.getString("env.USER", "unknownClient"); String pathPrefix = props.getString(VoldemortBuildAndPushJob.PUSH_CDN_PREFIX, "/tmp/VoldemortBnP/"); pathPrefix = pathPrefix.endsWith("/") ? pathPrefix : pathPrefix + "/"; String cdnDir = source.replaceAll(".*://.*?(?=/)", cdnURL + pathPrefix + username); FileSystem cdnRootFS = getRootFS(cdnDir); Path from = new Path(source); Path to = new Path(cdnDir); if (!prereqSatisfied(cdnURL)) { warn("\"other_namenodes\" does not contain the CDN cluster address " + cdnURL); warn("The following steps will fail soon, and distcp will be skipped!"); warn("Please add/append \"" + cdnURL + "\" to the \"other_namenodes\" attribute in your job specification."); } deleteDir(cdnRootFS, cdnDir); runDistcp(from, to); deleteDirOnExit(cdnRootFS, cdnDir); addPermissionsToParents(cdnRootFS, cdnDir); source = cdnDir; info("Use CDN HDFS cluster: " + source); } catch (Exception e) { warn("An exception occurred during distcp: " + e.getMessage()); warn("Use original HDFS cluster: " + source); } info("####################################"); info(" End of Distcp"); info("####################################"); } }</code><technical_language>I assume distcp multiple prod clusters run in parallel, logs for clusters interleave together. is one-line-logging time</technical_language>

Target:
public void run() throws Exception { info("############### Distcp ###############"); long startTime = System.currentTimeMillis(); try { String cdnURL = pickCDN(); if (!cdnURL.isEmpty()) { info("CDN: " + cdnURL); String storeName = props.getString(VoldemortBuildAndPushJob.PUSH_STORE_NAME).trim(); String pathPrefix = props.getString(VoldemortBuildAndPushJob.PUSH_CDN_PREFIX).trim(); if (!pathPrefix.startsWith("/")) { throw new RuntimeException(VoldemortBuildAndPushJob.PUSH_CDN_PREFIX + " must start with '/': " + pathPrefix); } pathPrefix = removeTrailingSlash(pathPrefix); String cdnDir = cdnURL + pathPrefix + "/" + storeName + extractPathFromUrl(source); cdnTargetFS = getTargetFS(cdnDir); Path from = new Path(source); Path to = new Path(cdnDir); if (!prereqSatisfied(cdnURL)) { warn("Please add/append \"" + cdnURL + "\" to the \"" + OTHER_NAMENODES + "\" attribute in your job specification."); throw new RuntimeException("\"" + OTHER_NAMENODES + "\" does not contain the CDN cluster address " + cdnURL); } deleteDir(cdnTargetFS, cdnDir); runDistcp(from, to); deleteDirOnExit(cdnTargetFS, cdnDir); addPermissionsToParentDirs(cdnTargetFS, cdnDir, pathPrefix); source = cdnDir; info("Will use data on CDN HDFS cluster: " + source); } } catch (Exception e) { warn("An exception occurred during distcp: ", e); warn("Will use data on original HDFS cluster: " + source); long duration = System.currentTimeMillis() - startTime; info("############# End of Distcp (FAILED) ############ (duration: " + duration/1000 + "s)"); throw new RuntimeException("An exception occurred during distcp", e); } long duration = System.currentTimeMillis() - startTime; info("############# End of Distcp ########### (duration: " + duration/1000 + "s)"); }