Merge branch 'mbm-redesign'
Revert "Revert "Various misc changes that still need additional testing. (This commit might be reverted pending additional testing and development in another branch.)""

This reverts commit d4f5ba21df2c361f41425ecad87e37a5570dde42.

Conflicts in pydpiper_apps/minc_tools/registration_chain.py were manually resolved.

NOTE: Having to revert the revert is ridiculous. I'm not sure why this didn't work properly the first few times w/r/t branching and merging. FAIL.
Misc small changes:
1. Removed if statement in minctracc atom that is no longer necessary or used.
2. Switched ordering of two arguments in minctracc call for consistency.
3. Fixed error with how names of new transforms are set.
Fixed error in registerVolume so that transform names will be set properly for groups where name = index.
Another crack at Issues #27,30,36:
1. Removed unnecessary returnEven from application (passed to pipelineDaemon)
2. Updated some logging messages for clarity.
3. Added buffer flush in launchServer when clients disconnected.
4. Updates to pipelineDaemon:
	a. Removed p.join() since this will happen immediately (and shouldn't) for queue=sge.
	b. Removed unnecessary while loop
	c. Added process.join() before returning to calling function
	d. Re-organized try/except for clarity
5. Fixed serverShutdownCall so it no longer takes unnecessary and ridiculous argument. Good-bye forever, hilarious code.
6. Added milliseconds to log file names so multiple executors launched at once no longer have the same log names.
Fixed bugs in setStageFailed
1. dfs_successors changed dfs_successor. This was an issue previously--I think it might have to do with python versioning in combo with networkx versioning.
   Stay tuned for this to pop-up again as versions evolve. Should be easily fixable.
2. index was being re-added to queue over and over, as opposed to successors. Whoops.
Addition of pairwise_nlin.py and some general minc_tools/framework changes:

1. Addition of pairwise_nlin.py to do pairwise nonlinear calculations. Relevant change made to setup.py.
2. Moved setup_logger() to application class. Can be overwritten, but I wanted to have a base function defined to avoid so much duplication.
	a. Existing files impacted: application.py, MAGeT.py, MBM.py
3. Created option groups to avoid duplication of command line options. Addition of option_groups.py file
	a. Existing files impacted: MBM.py, registration_chain.py, stats_tools.py
	b. NOTE: stats specific options defined in stats_tools.py. This will be the general paradigm we use going forward. (Options defined in classes where they are needed/used.)
4. Moved temporary MBM interface functions, previously in registration_chain.py, to their own file: old_MBM_interface_functions.py
5. Modified getLsq6AndXfms to getXfms only. getLsq6Files now its own functions.
6. Updated CalcStats class so calculations not done automatically in __init__. Now down in fullStatsCalc() function. Additional changes forthcoming that will further modularize this code.
Removed e.clear() in launchServer function.

This should have been removed as part of commit d724ee10e1491 since while e.is_set() loop was also removed.
Several minor changes to eliminate repeated code and/or fold into AbstractApplication class.
1. Removed backup-dir as option and replaced with output-dir.
2. Backups now placed in output-dir by default.
3. setup_backupDir now setup_directories, sets up backup and output directories, all in AbstractApplication class.
4. Removed output-dir from MAGeT.py and pipeline-dir from MBM, registration_chain and pairwise_nlin, since can now use output-dir.
5. Moved self.reconstructCommand() to AbstractApplication class.
6. In subclasses, removed options=self.options and args=self.args to make code-writing more straightforward for other users down the road.
Updated all executables so that all options are part of an option group for clarity of reading --help.
Removed unused import statements.
Two changes:
1. Updated setLastBasevol() so that if no argument is specified, it uses the original file used to instantiate the RegistrationPipeFH class
2. Updated stats_tools so that if no linearXfm/scaling factor is present, only jacobians (not scaled) are calculated.

NOTE: This is my second attempt at this checkin, since pushing to github last time didn't work.
Changes to how masks are initialized + various small option group changes.

1. initializeInputFiles now takes an optional mask directory as an argument. If the directory is specified, there are three options:
   a. If one mask is in the directory, file handlers are initialized with the same mask for all brains.
   b. If multiple masks are in the directory, they are matched with the appropriate brain according to their name. (Names must be similar.)
   c. If the number of masks in the directory does not match the number of brains, the program exits with an error message in the log file.
This masking logic was removed where replicated in registration_chain and pairwise_nlin.

2. Various changes to option groups:
   a. Created a new option group: General registration options, which includes pipeline-name, registration-method and mask-directory
   b. Moved init-model to MBM option group.

3. Fixed tabbing error in getXfms function that was causing registration chains to crash.
Updates to stats calculation so that full displacement can be used (instead of pure nonlinear).
If using this option, scaled jacobians can no longer be calculated, so appropriate changes have been made.
Updated getXfms in old_MBM_interface_functions to correctly handle lsq12 option.
Updated command line options in pairwise_nlin.py and registration_chain.py accordingly. (Also, to add consistency/clarity in how they are used.)
Unrelated to above: modification in pairwise_nlin.py to use full displacement instead of pure nlin component. (This may be removed later.)
Comment in stats class regarding calculation of scaled jacobian--will likely need to change handling later.
Changes to allow for lsq12 registration in registration_chain and pairwise_nlin when the registration method is mincANTS.

1. Updated minctracc atom so that default transforms will be set appropriately for lsq6 and lsq12 registrations if there is no last transform for source/target.
2. Wrote LSQ12 module to do basic lsq12 registrations as currently done in MICe-build-model. currently in minc_modules.py but may be moved.
3. Updated registration_chain.py and pairwise_nlin.py to use lsq12 if the registration method is mincANTS.
4. Added -tol = 0.0001 for -lsq6 and -lsq12 in minctracc atom. (Previously default was used.)
Fixed issue from last check-in: need to resample files after lsq12 registration prior to running mincANTS.
Related changes:
Created LSQ12ANTSNlin module that does lsq12 registration, resampling of files and mincANTS call.
Includes associated changes to: MAGeT_modules, registration_chain and pairwise_nlin

Other minor changes and bug fixes:
1. Moved createMask out of HierarchicalMinctracc. Changed argument to defaultDirectory. Default directory changed to tmp.
2. LSQ12 module. Changed function name from buildCommand to buildPipeline for naming consistency with other modules.
3. Registration_chain.py now uses full stats calc rather than removing lin_from_nlin.
4. setLastBasevol() with no argument now uses original input for each group instead of original input file. (This was problematic for MAGeT with new LSQ12ANTSnlin module.)
5. Fixed error in stats_tools hwere transform and inverseXfm not being set properly in statsGroup. Added setupXfms function to __init__
Fixed error in LSQ12ANTSNlin where input files were not added as such, so dependencies were not being calculated properly.
Fixed bug in old_MBM_interface_functions that caused an indexing error if the final timepoint was used as the average.
Updated registration_chain.py so that stats are now calculated from each subject to the average time point. This is in addition to the existing calculations between pairs.
1. All stats calculations moved to concatAndResample, which was renamed to LongitudinalStatsConcatAndResample.
	a. Changed from function to class to better handle increased complexity.
	b. Additional array added to keep track of xfms to average (in addition to common space).
	c. Added calculation of stats from each time point to average, then resampled to common space. Final time point handled separately and only if not average.
	d. General cleaning up of code and creating of functions to eliminate duplicate functionality.
	e. Looping over blurs now happens in resampleToCommon.

2. Moved transform and inverseXfm out of statsGroup, since they aren't really necessary there.
3. Removed subjectStats dictionary from registration_chain. Again, not really necessary.
4. Updated minc_displacement defaults so that if there is no specified xfm, it gets the last xfm between input and target. Removed targetFH as necessary file in input array.
Misc rearranging of files.
Moved HierarchicalMinctracc classes to their own .py file.
Moved LongitudinalStatsConcatAndResample, concatXfm and resampleToCommon to minc_modules.py
Updated calls to above classes in accordance with their new locations.
Fixed error in pairwise_nlin introduced in last check-in. (resampleToCommon now takes array of blurs instead of a single blur)