Pythonスクリプトのプロファイリング

# 準備

```
easy_install line_profiler
```

# cProfile

- コード全体をプロファイリング
- どの関数が一番時間を食っているかを明らかにする

```
python -m cProfile -s cumulative chi2.py > profile
less profile
```

# cProfileの結果

```
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.002    0.002    7.516    7.516 chi2.py:3(<module>)
        1    0.000    0.000    6.099    6.099 chi2.py:101(chi2)
     4959    0.229    0.000    3.511    0.001 chi2.py:11(detect_step)
       29    0.001    0.000    3.291    0.113 chi2.py:84(calculate_border_bestfit)
      435    0.008    0.000    3.289    0.008 chi2.py:35(detect_step_all_ranges)
    76661    0.219    0.000    2.738    0.000 fromnumeric.py:2830(var)
        1    0.000    0.000    2.538    2.538 chi2.py:53(plot)
    76661    1.215    0.000    2.519    0.000 _methods.py:77(_var)
        1    0.000    0.000    2.421    2.421 pyplot.py:138(show)
        1    0.000    0.000    2.421    2.421 backend_bases.py:140(__call__)
        1    0.000    0.000    2.417    2.417 backend_macosx.py:28(mainloop)
        1    2.323    2.323    2.417    2.417 {matplotlib.backends._macosx.show}
        1    0.001    0.001    1.251    1.251 pyplot.py:17(<module>)
```

- 結果を見ると、chi2関数が一番時間を食っている
- chi2関数をよく調べてみる

# Line profile

- chi2関数に@profileを付加

```py:chi2.py
@profile
def chi2(trajectory, t):
    s = []
    border_bestfit_best = []
    max_s = 0.0

    for nstep in range(1, 30):
        border_bestfit = calculate_border_bestfit(trajectory, nstep)
        border_counter = calculate_border_counter(trajectory, border_bestfit)
        s = calculate_chi2(trajectory, border_counter) / calculate_chi2(trajectory, border_bestfit)
        if max_s < s:
            max_s = s
            border_bestfit_best = border_bestfit

    # Calculate average trajectory
    average_trajectory = [np.average(trajectory[border_bestfit_best[i]:border_bestfit_best[i+1]])
                          for i in range(0, len(border_bestfit_best) - 1)]

    # Plot
    plot(trajectory, border_bestfit_best, average_trajectory)

    # Calculate step size
    step_size = [average_trajectory[i+1] - average_trajectory[i]
                 for i in range(0, len(average_trajectory)-1)]

    return step_size
```

- kernprof.pyを使ってlprofファイルを作成

```
kernprof.py -l chi2.py
```

# Line profilerの結果

```
python -m line_profiler chi2.py.lprof
```

```
Timer unit: 1e-06 s

Total time: 6.75531 s
File: chi2.py
Function: chi2 at line 101

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   101                                           @profile
   102                                           def chi2(trajectory, t):
   103         1            3      3.0      0.0      s = []
   104         1            1      1.0      0.0      border_bestfit_best = []
   105         1            1      1.0      0.0      max_s = 0.0
   106
   107        30           23      0.8      0.0      for nstep in range(1, 30):
   108        29      4188417 144428.2     62.0          border_bestfit = calculate_border_bestfit(trajectory, nstep)
   109        29       310665  10712.6      4.6          border_counter = calculate_border_counter(trajectory, border_bestfit)
   110        29        46667   1609.2      0.7          s = calculate_chi2(trajectory, border_counter) / calculate_chi2(trajectory, border_bestfit)
   111        29           39      1.3      0.0          if max_s < s:
   112         7            5      0.7      0.0              max_s = s
   113         7            7      1.0      0.0              border_bestfit_best = border_bestfit
   114
   115                                               # Calculate average trajectory
   116         1            1      1.0      0.0      average_trajectory = [np.average(trajectory[border_bestfit_best[i]:border_bestfit_best[i+1]])
   117        12          289     24.1      0.0                            for i in range(0, len(border_bestfit_best) - 1)]
   118
   119                                               # Plot
   120         1      2209171 2209171.0     32.7      plot(trajectory, border_bestfit_best, average_trajectory)
   121
   122                                               # Calculate step size
   123         1            2      2.0      0.0      step_size = [average_trajectory[i+1] - average_trajectory[i]
   124        11           18      1.6      0.0                   for i in range(0, len(average_trajectory)-1)]
   125
   126         1            0      0.0      0.0      return step_size
```

- 行ごとにかかった時間が表示されている
- chi2関数の計算時間の62%はcalculate_border_bestfit関数の計算に使用されている
- 今度は、calculate_border_bestfit関数をline profilerにかける
- この作業を繰り返すことでどの部分の計算に時間を浪費しているかを明らかにする