ContinuumIO/hexrd — pull request #2 by ddale (2014-11-06 15:57:58+00:00)
Title: Basicopt

this pull request refactors the multiprocessing implementation in `fitgrains` to allow operation without multiprocessing, in order to enable profiling.

It also renames the profiling script, and modifies it in order to use the `fitgrains` module, which is the new home for the `pull_spots` routines that were optimized by continuum in `pull_spots_mp`. Results from processing 1 grain:

```
$ python instrumented-fit-grains.py -c 1 -p ../multiruby.yml
Using cfg file '../multiruby.yml'

Total processing time 68.40 seconds
9833069 function calls (9828738 primitive calls) in 68.399 seconds

Ordered by: cumulative time

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.003 0.003 68.380 68.380 fitgrains.py:198(fit_grains)
1 0.001 0.001 40.236 40.236 fitgrains.py:168(get_data)
1 7.602 7.602 38.570 38.570 fitgrains.py:43(get_frames)
1 0.000 0.000 28.108 28.108 fitgrains.py:401(run)
2 0.000 0.000 28.108 14.054 fitgrains.py:382(loop)
1 0.000 0.000 28.064 28.064 fitgrains.py:304(pull_spots)
1 0.260 0.260 28.064 28.064 xrdutil.py:3258(pullSpots)
494 5.816 0.012 21.086 0.043 gridutil.py:93(cellCentroids)
1440 19.470 0.014 19.470 0.014 {method 'nonzero' of 'numpy.ndarray' objects}
822136 0.726 0.000 15.263 0.000 {method 'mean' of 'numpy.ndarray' objects}
822136 6.186 0.000 14.537 0.000 _methods.py:53(_mean)
1441 6.830 0.005 11.253 0.008 detector.py:1193(read)
1788596 6.396 0.000 6.399 0.000 {numpy.core.multiarray.array}
871599 4.705 0.000 4.705 0.000 {method 'reduce' of 'numpy.ufunc' objects}
494 3.166 0.006 4.421 0.009 gridutil.py:62(cellConnectivity)
1442 0.014 0.000 3.917 0.003 detector.py:277(frame)
822136 1.258 0.000 1.858 0.000 _methods.py:43(_count_reduce_items)
895521 0.581 0.000 1.741 0.000 numeric.py:464(asanyarray)
1 0.000 0.000 1.652 1.652 coreutil.py:152(initialize_experiment)
1 0.000 0.000 1.525 1.525 experiment.py:114(__init__)
1 0.000 0.000 1.522 1.522 material.py:305(loadMaterialList)
6 0.000 0.000 1.521 0.253 material.py:67(__init__)
6 0.000 0.000 1.521 0.253 material.py:105(_readCfg)
6 0.000 0.000 1.520 0.253 material.py:203(_set_sgnum)
6 0.000 0.000 1.520 0.253 material.py:149(_newPdata)
6 0.000 0.000 1.463 0.244 crystallography.py:500(__init__)
6 0.000 0.000 1.460 0.243 crystallography.py:553(__calc)
6 0.008 0.001 1.459 0.243 crystallography.py:687(makePlaneData)
830 0.016 0.000 1.397 0.002 symmetry.py:313(applySym)
830 0.351 0.000 1.118 0.001 matrixutil.py:538(findDuplicateVectors)
1728 0.105 0.000 0.928 0.001 distortion.py:20(GE_41RT)
1711095/1711094 0.790 0.000 0.790 0.000 {isinstance}
1710 0.061 0.000 0.785 0.000 distortion.py:9(newton)
1441 0.011 0.000 0.471 0.000 detector.py:1294(__readNext)
5153 0.334 0.000 0.334 0.000 distortion.py:51()
5153 0.331 0.000 0.331 0.000 distortion.py:56()
825622/825619 0.328 0.000 0.328 0.000 {issubclass}
1 0.008 0.008 0.322 0.322 xrdutil.py:3115(simulateGVecs)
20366 0.045 0.000 0.320 0.000 shape_base.py:230(hstack)
1441 0.011 0.000 0.306 0.000 threading.py:726(start)
946 0.083 0.000 0.301 0.000 transforms.py:100(anglesToGVec)
47324 0.103 0.000 0.265 0.000 shape_base.py:8(atleast_1d)
1440 0.035 0.000 0.256 0.000 coo.py:116(__init__)
1441 0.010 0.000 0.243 0.000 threading.py:602(wait)
15718 0.123 0.000 0.238 0.000 shape_base.py:792(tile)
1441 0.017 0.000 0.231 0.000 threading.py:308(wait)
7209 0.207 0.000 0.207 0.000 {method 'acquire' of 'thread.lock' objects}
24945 0.019 0.000 0.191 0.000 {method 'sum' of 'numpy.ndarray' objects}
830 0.102 0.000 0.184 0.000 rotations.py:474(rotMatOfQuat)
26078 0.016 0.000 0.180 0.000 _methods.py:31(_sum)
946 0.015 0.000 0.179 0.000 transforms.py:796(makeEtaFrameRotMat)
494 0.004 0.000 0.168 0.000 measurements.py:46(label)
452 0.044 0.000 0.167 0.000 measurements.py:1231(center_of_mass)
494 0.150 0.000 0.158 0.000 {scipy.ndimage._ni_label._label}
4217 0.066 0.000 0.153 0.000 index_tricks.py:251(__getitem__)
4437 0.132 0.000 0.142 0.000 xrdutil.py:3247(_coo_build_window)
1440 0.051 0.000 0.131 0.000 coo.py:236(_check)
32767 0.105 0.000 0.105 0.000 {numpy.core.multiarray.concatenate}
1482 0.064 0.000 0.104 0.000 gridutil.py:8(cellIndices)
71524 0.096 0.000 0.096 0.000 {method 'reshape' of 'numpy.ndarray' objects}
1892 0.036 0.000 0.095 0.000 transforms.py:719(unitVector)
1808 0.003 0.000 0.094 0.000 measurements.py:540(sum)
1710 0.008 0.000 0.091 0.000 transforms_CAPI.py:84(gvecToDetectorXY)
```