Ticket #3308 Fix Installer breakage from name changes
Merge branch 'V7_8-branch' into V7_9_2-branch
Improve DAGMAN_MAX_JOBS_IDLE documentation #3342

Try to eliminate ambiguity described in following abridged conversation
from #distcomp on 27 November 2012:

"20:13:58 <ichesal> hey guys -- can someone answer a DAGMan question for
me? I have the DAGMAN_MAX_JOBS_IDLE=2500 in my config and I can confirm
that condor_dagman is seeing the setting:
20:14:02 <ichesal> 11/27/12 17:59:07 DAGMAN_MAX_JOBS_IDLE setting: 2500
20:14:56 <ichesal> I have a node in my DAG where the .sub file does a
"Queue 18797" call. I was under the impression the above setting would
cause DAGMan to actually chunk that up in to multiple clusters with no
more than 2500 jobs in it. But it's not. It's submitting all of the in
one go:
20:15:35 <ichesal> 11/27/12 17:59:23 Submitting Condor Node RunVAHA
job(s)...
20:15:35 <ichesal> 11/27/12 17:59:23 submitting: condor_submit -a
dag_node_name' '=' 'RunVAHA -a +DAGManJobId' '=' '89 -a DAGManJobId' '='
'89 -a
20:15:35 <ichesal> submit_event_notes' '=' 'DAG' 'Node:' 'RunVAHA -a
log' '=' '/mnt/X/run/Hedging/Production/AutoRunPackages/20121127/HedgingSubda
20:15:37 <ichesal> g.dag.nodes.log -a +DAGParentNodeNames' '=' '""
/mnt/X/run/Hedging/Production/submission/CHPAUTOSUBMITTER-20121127-175106PM_VAH
20:15:39 <ichesal> A_v0_3_1_IT_Test_/analysis.sub
20:15:41 <ichesal> 11/27/12 17:59:29 From submit: Submitting job(s)
20:15:43 <ichesal> <snip>
20:15:50 <bbockelm> Doesn't DAGMan only operate on individual jobs?
20:15:53 <ichesal> 11/27/12 18:06:41 From submit: 18797 job(s) submitted
to cluster 90.
20:16:14 <ichesal> What's an "individual job" though? I read that and
interpreted it as "procs in a cluster".
20:16:35 <ichesal> I'm running out of file descriptors very quickly once
those 18k jobs hit the queue.
20:17:49 <ichesal> hmm. So I need to break it up in 18000 .sub files?
20:18:32 <bbockelm> maybe.  I'm a bit amateur on DAGMan.
20:18:38 <bbockelm> there might be something clever which can be done.
20:20:31 <ichesal> hmmm.
20:20:41 <bbockelm> Anyhow, I definitely recall having issues with
"queue 5000" - type statements.
20:21:03 <bbockelm> also, if one proc in the cluster fails, it would
kill and resubmit the whole cluster.
20:23:29 <ichesal> brian, I think you're correct. This is alluding to
it: http://research.cs.wisc.edu/htcondor/manual/v7.7/
2_10DAGMan_Applications.html#SECTION003107400000000000000
20:25:08 <ichesal> i'm going to say the docs for DAGMAN_MAX_JOBS_IDLE
are not very good at describing the level at which it's limiting things!
20:25:13 <bbockelm> such as the one I described above… one job in a
cluster of a million fails, the whole cluster is resubmitted.
21:22:09 <ichesal> brian, the more I look at it, the more it makes sense
the DAGMan limit is on the cluster level. I guess I was hoping there was
some cluster cracking voodoo in there. oh well.
21:57:29 <ichesal> yup. chunking up the 18k job submission in to
multiple sub's and submitting them as siblings fixed it all. thanks
brian! helped to talk it out."

Make it clear that no "cluster cracking voodoo" occurs
Merge branch 'V7_8-branch' into V7_9_2-branch
Fix for incorrect X_RUNS_HERE default, now TRUE again
Clarified the definitions of DAGManJobId, DAGManNodesLog, and
DAGManNodesMask; fixed a typo.
Merge branch 'V7_9_2-branch' of /p/condor/repository/CONDOR_SRC into V7_9_2-branch
Very simple fix for calling os._exit vs sys.exit when -h is called. On
RHEL5 sys.exit raises an exception which then calls usage.
Merge branch 'V7_8-branch' into V7_9_2-branch
minor edits
===GT=== #3311
Merge branch 'V7_9_2-branch' of /p/condor/repository/CONDOR_SRC into V7_9_2-branch
minor version history item edit to add pronouns and end the sentence
with punctuation.
===GT=== #3316
Clarify the grid type EC2 addition of euca3:// and euca3s:// protocols,
as well as the version history item that describes them.
===GT=== #2974
Ticket #3349 Cleanup invalid -format handling for condor_history
Merge branch 'V7_8-branch' into V7_9_2-branch
edit to correct broken links or remove reference to them in manual
===GT=== #3340
correct and update many links related to grid computing
===GT=== #3340
Merge branch 'V7_8-branch' of /p/condor/repository/CONDOR_SRC into V7_8-branch
Merge branch 'V7_8-branch' into V7_9_2-branch
Revert "Ticket #3349 Cleanup invalid -format handling for condor_history"

This reverts commit 812716faa6cee4721c9f58871ddc59a502abaaf5.