scontrol show job - Correct NumNodes value

It has been changed to improve the calculated value for pending
jobs and use the actual node count value for jobs that have been
started (including suspended, completed, etc.)
bug 549
BGQ - Fix issue if user runs multiple sub-block jobs inside a multiple
midplane block that starts on a higher coordinate than it ends (i.e if a
block has midplanes [0010,0013] 0013 is the start even though it is
listed second in the hostlist).
BGQ - Add midplane to the total_cnodes used in the runjob_mux plugin
for better debug
BGQ - Fix uninitialized variable from commit
0c00d359f27c8c88bf7a8af9c5f5fee8c6d9fb15
BGQ - better debug in both selecting steps and launching them in the
runjob_mux.
BGQ - change some "DEBUG" messages to "TRACE"
BGQ - Fix incorrect num of variables in print statement
Change SLURM to Slurm
Clearer explanation about AccountingStorageEnforce
Update AllocNodes paragraph in slurm.conf.5.
Update META for v2.6.5 tag
Update news to start 2.6.6
Add an exception handler in case the test log file does not exist.
Minor sbatch man page clarification
Fix sched/backfill bug that could starve jobs

Hello,

I think I found another bug in the code (I'm using 2.6.3 but I checked the 2.6.5 and 14.03 versions and it's the same there).

In file sched/backfill/backfill.c:

1)
_add_reservation function, from lines 1172:

if (placed == true) {
        j = node_space[j].next;
        if (j && (end_reserve < node_space[j].end_time)) {
                /* insert end entry record */
                i = *node_space_recs;
                node_space[i].begin_time = end_reserve;
                node_space[i].end_time = node_space[j].end_time;
                node_space[j].end_time = end_reserve;
                node_space[i].avail_bitmap =
                        bit_copy(node_space[j].avail_bitmap);
                node_space[i].next = node_space[j].next;
                node_space[j].next = i;
                (*node_space_recs)++;
        }
        break;
}
I draw a picture with `node_space` state after 2 iterations (see attachment).

In case where the new reservation is fully inside another reservation,
then everything is OK.
But if the new reservation spans multiple existing reservations then the `end entry record` is not created.
This is because only the newly created `start entry record` is checked.

Easy fix would be to change the if into a loop, for example:

if (placed == true) {
    while((j = node_space[j].next) > 0) {
        if (end_reserve < node_space[j].end_time) {
           //same as above
           break;
        }
    }
    break;
}

2)
You could also change line 612:
        node_space = xmalloc(sizeof(node_space_map_t) *
                             (max_backfill_job_cnt + 3));
To `(max_backfill_job_cnt * 2 + 1)` , since each reservation can add at most two entries (check at line 982 should never execute). At the moment, in a worst case scenario this only checks half of the max_backfill_job_cnt.

NOTE: However this is all based on the assumption, that it is not done on purpose to speed up the calculations and trading some of the accuracy (especially point 2).

Best regards,
Filip Skalski
Correct updated info in troubleshooting guide
Fix a couple of typos in web page
Correct job RunTime if requeued from suspend state

Without this patch, the job's RunTime includes its RunTime from
before it's prior suspend (i.e. the job's full RunTime rather than
just the RunTime of the requeued job).
Reset job priority on manual resume

If a job is explicitly suspended, its priority is set to zero.
This resets the priority when requeued and also documents that
if the job is requeued (e.g. due to a node failure), then it
is placed in a held state.
Add link to Lua hostlist module in download web page