悪の組織（架空）と学ぶ 分類性能評価指標

例えば感染症の検査法のように「判定」する問題は、機械学習分野では「分類」という言い方をしますが、日常生活でも非常によく登場します。その判定法、あるいは分類法の性能をどう捉えるかは非常に難しい問題であり、日常的によく見られるにもかかわらず、その考え方は一般に普及しているとはとても言えません。その性能指標には様々なものがありますが、それぞれの関係は私も詳しく分かっていませんでしたので、悪の組織（架空）と一緒に計算機実験を行ってみることにしました。

# 第１話「混同行列と十人の被験者」の巻

簡単な例として、次のように、１０人の被験者に登場していただき、ある感染症に感染しているか否かを検査するケースを想定しましょう。

ここで Y は「感染者なら1、非感染者なら0」のベクトルとします。

同様に Y_pred は、検査により判定された Y の予測値、つまり「陽性（感染者であると判定された）なら1, 陰性（非感染者であると判定された）なら0」のベクトルとします。


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [1, 1, 1, 0, 1, 1, 0, 0, 0, 0]
```

「感染・非感染」と「陽性・陰性」を混同している方もけっこう多くおられますが、それらは別概念であり、完全一致するとは限らないことに注意しましょう。この結果から、被験者を４つのグループに分類できます。

- TP (True Positive, 真陽性)： 陽性（感染者であると判定された）で、かつ、本当に感染者である。
- FN (False Negative, 偽陰性)：陰性（非感染者であると判定された）だが、本当は感染者である。
- FP (False Positive, 偽陽性)：陽性（感染者であると判定された）だが、本当は非感染者である。
- TN (True Negative, 真陰性）：陰性（非感染者であると判定された）で、かつ、本当に非感染者である。

上の Y と Y_pred から、 TP, FN, FP, TN の数をカウントする関数を作ってみましょう。


```python
def yypred_to_confusion_table(Y, Y_pred):
    tp = fn = fp = tn = 0
    for y, y_pred in zip(Y, Y_pred):
        if y == 1 and y_pred == 1:
            tp += 1
        elif y == 1 and y_pred == 0:
            fn += 1
        elif y == 0 and y_pred == 1:
            fp += 1
        else:
            tn += 1

    return [tp, fn, fp, tn]
```

この関数の実行結果は次のようになります。


```python
confusion_table = yypred_to_confusion_table(Y, Y_pred)
confusion_table
```




    [3, 1, 2, 4]



これを「混同行列」という名の表で表現してみましょう。次のような関数を作ってみます。


```python
import numpy as np
import pandas as pd


def draw_confusion_table(confusion_table):
    return pd.DataFrame(
        np.array(confusion_table).reshape(2, 2).T,
        columns=["感染", "非感染"],
        index=["陽性", "陰性"],
    )
```

この関数の実行結果は次のようになります。


```python
draw_confusion_table(confusion_table)
```




<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>感染</th>
      <th>非感染</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>陽性</th>
      <td>3</td>
      <td>2</td>
    </tr>
    <tr>
      <th>陰性</th>
      <td>1</td>
      <td>4</td>
    </tr>
  </tbody>
</table>



混同行列をグラフ表示してみましょう。次のような関数を作ってみます。


```python
import matplotlib.pyplot as plt


def depict_confusion_table(table):
    tp, fn, fp, tn = table
    plt.title("TP, FN, FP, TN = {}, {}, {}, {}".format(tp, fn, fp, tn))
    plt.bar(["Infected", "Noninfected"], [-fn, -tn])
    plt.bar(["Infected", "Noninfected"], [tp, fp])
    plt.ylabel("Negative / Positive")
    plt.grid()
    plt.show()
```


```python
depict_confusion_table(confusion_table)
```
![output_12_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/fdbb608b-d423-d4ec-7094-ce453d7f8a57.png)


      


上の図で、横軸は感染者（Infected）と非感染者（Noninfected）を表します。

縦軸は、陰性（Negative）の人の数だけ負の方向に青っぽい色の縦向き棒グラフが、陽性（Positive）の人の数だけ正の方向にオレンジ色の縦向き棒グラフが表示されるようにしました。

このグラフは、今勝手に作った人工データに過ぎませんが、ここから何が言えるか考えてみましょう。この検査方法は完璧な検査方法ではなく、感染者であっても陰性と判定されたり、非感染者であっても陽性と判定されたりしてしまうということです。

# 第２話「混同行列と完璧な検査法」の巻

それでは、もしも「完璧な検査法」があったとしたら、どのようなグラフになるでしょうか。そんな完璧な検査法など、実在しないとは思いますが、あくまで仮定の話として、作ってみましょう。このようになります。


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]

depict_confusion_table(yypred_to_confusion_table(Y, Y_pred))
```

![output_14_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/d6725861-1a54-76c8-1997-2dfbc46e926b.png)

    
    


ここで、上記の２つのグラフにおいて、感染者の数および非感染者の数は変わっていないことに注意してください。感染者・非感染者の数が変化するのではなくて、それを陽性と判定するのか、陰性と判定するのかが変わるということです。

# 第３話「悪の組織、検査結果を全て陰性とする」の巻

そこにある日、謎の悪の組織（架空）が現れました。その悪の組織（架空）は、「検査を行ったフリをして、実際には検査を行わず、検査結果を常に "陰性" ということにして、検査費用だけを請求する」などというとんでもない悪事を働きました。そのような場合、どのようなグラフになるか作ってみましょう。


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

depict_confusion_table(yypred_to_confusion_table(Y, Y_pred))
```

![output_16_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/a4b2f514-0776-13f3-ecd6-016e7f334fc6.png)

    
    


ここまでで、混同行列とそのグラフについて、その見方についてご理解いただけたかと思います。それでは、以下のお話では、悪の組織（架空）の検査法に立ち向かっていきましょう。

# 第４話「様々な視点で闘え、分類性能評価指標」の巻

悪の組織（架空）の検査法に立ち向かうため、分類性能評価指標が現れました。分類性能評価指標には多くの種類があります。列挙してみましょう。それぞれの分類性能評価指標の意味は、ここでは説明しません。気になる方は https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics などを参照してください。この記事は、分類性能評価指標の正確な意味を理解していなくても、「その挙動を何となく実感してもらう」ことを目的としています。


```python
from sklearn import metrics

measures = [
    ["Accuracy", metrics.accuracy_score],
    ["Balanced Accuracy", metrics.balanced_accuracy_score],
    ["Recall", metrics.recall_score],
    ["Precision", metrics.precision_score],
    ["Average Precision", metrics.average_precision_score],
    ["F1", metrics.f1_score],
    ["Jaccard", metrics.jaccard_score],
    ["Cohen's Kappa", metrics.cohen_kappa_score],
    ["MCC", metrics.matthews_corrcoef],
]
```

これらの分類性能評価指標を全て計算する関数を作ってみましょう。


```python
import warnings


def depict_classification_metrics(Y, Y_pred):
    warnings.simplefilter("ignore")
    fig, axes = plt.subplots(
        ncols=2,
        nrows=1,
        figsize=(6, 3),
        gridspec_kw=dict(width_ratios=[2, 1], wspace=0.3, hspace=0.1),
    )
    plt.subplots_adjust(left=0, right=1, bottom=0, top=1)

    tp, fn, fp, tn = yypred_to_confusion_table(Y, Y_pred)
    axes[1].set_title("TP, FN, FP, TN = {}, {}, {}, {}".format(tp, fn, fp, tn))
    axes[1].bar(["Infected", "Noninfected"], [-fn, -tn])
    axes[1].bar(["Infected", "Noninfected"], [tp, fp])
    axes[1].set_ylabel("Negative / Positive")
    axes[1].grid()

    scores = []
    negative_score = False
    for measure_name, measure in measures[::-1]:
        score = measure(Y, Y_pred)
        if score < 0:
            negative_score = True
        scores.append([measure_name, score])

    axes[0].set_title("Classification metrics")
    axes[0].barh([x[0] for x in scores], [x[1] for x in scores])
    if negative_score:
        axes[0].set_xlim([-1, 1])
        axes[0].set_xticks(np.linspace(-1, 1, 21))
        for tick in axes[0].get_xticklabels():
            tick.set_rotation(90)
    else:
        axes[0].set_xlim([0, 1])
        axes[0].set_xticks(np.linspace(0, 1, 11))
    axes[0].set_xlabel("Score")
    axes[0].grid()
    plt.show()
```

この関数を用いて、最初に挙げた十人の被験者の例を図示すると、次のようになります。


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [1, 1, 1, 0, 1, 1, 0, 0, 0, 0]

depict_classification_metrics(Y, Y_pred)
```

![output_23_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/fac34fc1-fb9e-a767-4339-466c983cbc5e.png)

    
    


完璧な検査法の場合は、次のようになります。全ての分類性能評価指標の値が 1 になります。


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]

depict_classification_metrics(Y, Y_pred)
```

![output_25_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9f6d4599-7d84-4d00-af6f-525b1bcee2c8.png)

    
    


先程の悪の組織（架空）の「全ての結果を陰性とする」デタラメな検査法は、次のようになります。

あれ？ 0 になる分類性能評価指標もあれば、そこそこの値が出る分類性能評価指標もありますね？


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

depict_classification_metrics(Y, Y_pred)
```

![output_27_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/f2876d25-588e-2893-f3fd-7a3a4fbd14a7.png)

    
    


悪の組織（架空）は次に、「全ての結果を陽性とする」デタラメ検査法を用いました。そうすると分類性能評価指標は次のようになります。

ほうほう、デタラメな検査法でも、いい値が出てしまう分類性能評価指標がありますね。


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

depict_classification_metrics(Y, Y_pred)
```

![output_29_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/8f63561e-4ef3-d87d-035a-7d213276ffd3.png)

    
    


今度は、悪の組織（架空）は「真実と完全に真逆の結果を示す」恐るべき検査法を使ってきました。その場合の分類性能評価指標を見てみましょう。


```python
Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Y_pred = [0, 0, 0, 0, 1, 1, 1, 1, 1, 1]

depict_classification_metrics(Y, Y_pred)
```

![output_31_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/0836d414-a1b3-8466-8769-39ccf803bc1f.png)

    
    


以上の簡単な実験からだいたい把握できたように、多くの指標は、デタラメな検査法に対しても高いスコアを出してしまうことがあります。

個人的には、 Cohen's Kappa score と MCC score の２つが、次のような性質を持つので好きですね（個人の感想です）。
 - デタラメな判定ではゼロになりやすい
 - 真逆の判定ではマイナスの値を取る
 
 
# 第５話「悪の組織（架空）と６つのシナリオ」

ここからは、より詳細なシナリオを設定して、それらの場合に分類性能評価指標がどう変化するかを実験してみましょう。次のようなシナリオを用意しました。

- シナリオ1-1：「悪の組織、全ての結果を "陰性" とする」の巻
- シナリオ1-2：「悪の組織、全ての結果を "陽性" とする」の巻
- シナリオ2-1：「悪の組織、5割の確率で "陽性" とする」の巻
- シナリオ2-2：「悪の組織、3割の確率で "陽性" とする」の巻
- シナリオ3：「悪の組織、真逆の結果を示す検査法を使う」の巻
- シナリオ4-1：「感染者は絶対見逃さない！だが非感染者の5割を誤判定する」の巻
- シナリオ4-2：「感染者は絶対見逃さない！だが非感染者の3割を誤判定する」の巻
- シナリオ5-1：「非感染者は絶対陽性判定しない！だが感染者の5割を誤判定する」の巻
- シナリオ5-2：「非感染者は絶対陽性判定しない！だが感染者の3割を誤判定する」の巻
- シナリオ6-1：「感染者・非感染者関係なく３割を誤判定する」の巻
- シナリオ6-2：「感染者の１割・非感染者の３割を誤判定する」の巻
- シナリオ6-3：「感染者の３割・非感染者の１割を誤判定する」の巻

# シナリオ1-1：「悪の組織、全ての結果を "陰性" とする」の巻

悪の組織（架空）は、全ての結果を「陰性」とする検査法を使いました。そのような検査法に対して、実際の「感染・非感染」の比率が変化すると、分類性能評価指標がどのように変化するかみてみましょう。まずは、十人の被験者に対して、その「感染・非感染」の比率を変化させてみると、次のようになります。


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [0] * n_people

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/7190c4de-9fec-0433-c4fc-1a2d6ca1835e.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ee40c005-19a8-7803-4c7a-451989463664.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/705c6b33-773f-2d59-8172-37b34ece27c5.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/1f771096-b5ae-46e2-b45c-c0352a014972.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ab5768a2-6ed1-c9d4-22a1-866e1cfcc20e.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9081ebb0-9186-fc7e-384b-3933105ee649.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/78e8b979-4d90-00f8-803e-fd9bb8c6ba37.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9421f388-df26-a981-1b54-2ac7b3acbe42.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ef939c68-5fb2-b50c-3519-6f9f85b9b272.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/1ea0fa8b-c818-15db-1824-fe765280a563.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_34_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ed840c66-5bb7-6495-3419-2b31a3847452.png)


    


同様に、「感染・非感染」の比率を変えた時の分類性能評価指標の変化を、今度は百人の被験者に対して行ってみましょう。計算結果を score_record という変数に記録します。


```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [0] * n_people

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```

さすがに１００人もいると、先ほどまでのような図では大変なので、次のような関数を作成して、図示の仕方を工夫してみましょう。


```python
def depict_layer(score_record):
    warnings.simplefilter("ignore")
    fig, axes = plt.subplots(
        ncols=1,
        nrows=len(measures),
        figsize=(6, 6),
        gridspec_kw=dict(wspace=0.3, hspace=0.18),
    )
    plt.subplots_adjust(left=0, right=1, bottom=0, top=1)
    for i, (measure_name, measure) in enumerate(measures):
        axes[i].scatter(
            score_record["p_infected"],
            score_record[measure_name],
            c=score_record["p_infected"],
            cmap="coolwarm",
            label=measure_name,
        )
        axes[i].grid()
        axes[i].legend()
        axes[i].set_xlabel("p_infected")
        axes[i].set_xticks(np.linspace(0, 1, 11))
        if min(score_record[measure_name]) < 0:
            axes[i].set_ylim([-1.1, 1.1])
            axes[i].set_yticks([-1, -0.5, 0, 0.5, 1])
        else:
            axes[i].set_ylim([-0.1, 1.1])
            axes[i].set_yticks([0, 0.5, 1])

    plt.show()
```

score_record の記録を図示します。横軸は「感染・非感染」の比率、縦軸はそれぞれの評価指標のスコアの値です。プロットの色は、「感染・非感染」の比率に対応して色付けしてあります。


```python
depict_layer(score_record)
```


    
![output_40_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/e4e1cb9f-f11c-a3db-dfb6-3dd91a5acf6f.png)


    


デタラメな検査に対して、常にゼロを出力する指標が多い中、高いスコアを出力する指標もあり、それが「感染・非感染」比に依存することもよくわかりますね。

次の関数は、それぞれの評価指標間を比較するための分布図マトリクスを図示します。


```python
def depict_matrix(score_record):
    warnings.simplefilter("ignore")
    fig, axes = plt.subplots(
        ncols=len(measures),
        nrows=len(measures),
        figsize=(12, 12),
        gridspec_kw=dict(wspace=1, hspace=1),
    )
    plt.subplots_adjust(left=0, right=1, bottom=0, top=1)
    for j, (measure_name1, measure1) in enumerate(measures):
        for i, (measure_name2, measure2) in enumerate(measures):
            axes[i, j].scatter(
                score_record[measure_name1],
                score_record[measure_name2],
                c=score_record["p_infected"],
                cmap="coolwarm",
            )
            axes[i, j].grid()
            # if i == len(measures) - 1:
            axes[i, j].set_xlabel(measure_name1)
            # if j == 0:
            axes[i, j].set_ylabel(measure_name2)
            if min(score_record[measure_name1]) < 0:
                axes[i, j].set_xlim([-1, 1])
                axes[i, j].set_xticks([-1, -0.5, 0, 0.5, 1])
            else:
                axes[i, j].set_xlim([0, 1])
                axes[i, j].set_xticks([0, 0.5, 1])
            if min(score_record[measure_name2]) < 0:
                axes[i, j].set_ylim([-1, 1])
                axes[i, j].set_yticks([-1, -0.5, 0, 0.5, 1])
            else:
                axes[i, j].set_ylim([0, 1])
                axes[i, j].set_yticks([0, 0.5, 1])
    plt.show()
```

各指標間の関係は次のように図示されます。プロットの色は、先ほどと同様、「感染・非感染」の比率に対応して色付けしてあります。


```python
depict_matrix(score_record)
```


    
![output_44_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/89804d8e-0155-8743-bf81-b5cd153b2a6d.png)


    


# シナリオ1-2：「悪の組織、全ての結果を "陽性" とする」の巻

今度の悪の組織（架空）は、先ほどと真逆で、全ての結果を「陽性」とする悪徳検査法を用いました。まず十人の被験者の場合は次のようになります。


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [1] * n_people

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/360d73b5-4c78-2d0e-c359-33af39205124.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/de0779b3-065c-5a5b-364a-0736d7b0f294.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/eb7cb0ed-cda0-c10d-6218-d16ff2b450e3.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/f144f3fe-4028-f5ee-aa16-5e9086e43030.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/eaf804b3-eaa2-9705-6d71-feb2f5e376e7.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/29e51a58-f8a9-042d-4187-532a9de348a3.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/3dbe36ac-5e62-d918-2aa9-420bfb7d1c74.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9291aa25-3985-bd24-5e69-9db0a3d16674.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/08a48059-2919-62eb-223c-53624d022553.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_46_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/06314b1c-9ffa-dbc8-f409-e65569a0d048.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
    
![output_46_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/465b681a-0f8d-d4b8-718b-7b263304a698.png)




多くの評価指標が高いスコアを示している中で、 Cohen's Kappa と MCC だけは常にゼロなのが印象的ですね（個人の感想です）。

次は１００人に対して「感染・非感染」比率を変えて評価指標の変化を計算してみましょう。


```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [1] * n_people

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_49_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/8307db24-a136-15cc-9ccd-e8b7fa3b58fc.png)


    



```python
depict_matrix(score_record)
```


    
![output_50_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/730cc877-9f9d-f34e-1367-a1bc7af86457.png)


    


感染率の増加に伴って増加するスコア同士であっても、増加の仕方が異なることが分かりますね。

# シナリオ2-1：「悪の組織、5割の確率で "陽性" とする」の巻

今度の悪の組織（架空）は、感染・非感染に関わらず、ランダムに５割の確率で陽性・陰性を決める検査法を使いました。


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [1 if j % 2 == 0 else 0 for j in range(n_people)]

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/bd2256c3-c685-5eff-324d-c03d4ecf9e49.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/4ed5a445-9e7b-d7b3-d9b4-7e01f71f45d8.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/6e6b1ea9-4fef-f8fe-8bda-6e2150000bd5.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/3890d372-fcaa-444f-2193-266918430616.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/90d2e837-47ac-2303-c45e-6e2187847b7c.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/162cfe7f-2d9f-7ebd-a549-ffdc6a57f841.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/fdb1c480-b345-cfa2-6b61-9e6f3028234c.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/4327cd14-0ec4-945e-e0e6-a395e8c55efc.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c3f9edd4-c6f3-6122-4c68-9570dab39c6a.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/4fe714e4-c4cd-eae3-2f9e-9aed5f9a4aae.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_53_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/7543da23-9cba-a9d3-66db-b3c128a6c54d.png)


    


やけに高いスコアを示す性能評価指標があるという点においては、これまでと同じですが、 Cohen's Kappa と MCC が、これまでとは異なる挙動を示していることにお気づきでしょうか？スコアがゼロになったり、小さいですが正の値を示したりを繰り返しています。このように、Cohen's Kappa と MCC は、青とオレンジが対称であればゼロになりますが、対称性が崩れれば非ゼロになるという特徴があり、ときどき「振動」するという特徴があります。

続いて百人の被験者に対して計算してみましょう。


```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [1 if j % 2 == 0 else 0 for j in range(n_people)]

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_56_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/3286c742-3503-3a06-aa00-92fbfdeb81b0.png)


    



```python
depict_matrix(score_record)
```


    
![output_57_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/a14e3c21-451b-ce4e-26d7-de960afbd8f0.png)


    


# シナリオ2-2：「悪の組織、3割の確率で "陽性" とする」の巻

悪の組織は、先ほどと類似ですが少しだけ違う検査法を使ってきました。性能評価指標の挙動も、先ほどとは異なる挙動を示しています。その挙動を眺めてみましょう。説明は、だんだん面倒になってきましたwww


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [1 if j % 3 == 0 else 0 for j in range(n_people)]

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/8bc839b5-90a8-a089-9fd3-d7bdb5222c70.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/a7b65f92-88af-3868-56a1-7676d9675634.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c86eaccd-bff0-3398-566c-e07a55f45932.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/0d0c44ff-93d4-15bc-48b1-d5a6945155e8.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/92cd1c87-6813-6dd8-5ee2-1d5963746cb2.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ee2b7f87-c78d-5b9e-f49a-8875c1ec5ebe.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/543c9a49-f4e9-5e49-3f74-cf0bf51ee930.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9739eff9-7a98-104a-a421-db93a403218b.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/eadce813-9172-b340-ff01-6e6be251606c.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/d46e60e1-0bea-bc46-597c-88879466a2cb.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 0, 0, 1, 0, 0, 1, 0, 0, 1]



    
![output_59_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/2b9814f3-117b-5052-ea7a-9adaf652877e.png)


    


Cohen's Kappa や MCC から、その値がゼロの時は「そんな検査法、使っても意味ねーよ」、マイナスの時は「そんな検査法、むしろ使わねーほうがいいよ」という叫びが聞こえてきそうです（個人の感想です）。


```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [1 if j % 3 == 0 else 0 for j in range(n_people)]

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_62_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/2445f883-195c-eacb-878e-9a36718b9008.png)


    



```python
depict_matrix(score_record)
```


    
![output_63_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/cfa90580-1fb4-09e9-c0db-4c615227cfd9.png)


    


# シナリオ3：「悪の組織、真逆の結果を示す検査法を使う」の巻

次の悪の組織（架空）は、真実とは真逆の結果を返す、ある意味凄すぎる検査法を使いました。そのときの性能評価指数の挙動を計算してみましょう。ああ、もう説明疲れた。


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [0 if i > j else 1 for j in range(n_people)]

    print("Y =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_65_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/e8b0d786-5842-104e-7fdf-0a5c220ef747.png)


    


    Y = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_65_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/0e4e35f9-6ef0-e984-6a60-1666091cd5da.png)


    


    Y = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_65_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/71c6dd05-a122-1d89-8030-42cee70d95df.png)


    


    Y = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 1, 1, 1, 1, 1, 1, 1]



    
![output_65_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/446845fd-ecdc-308c-1335-7e2257708a71.png)


    


    Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 1, 1, 1, 1, 1, 1]



    
![output_65_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/7c463bfc-46a1-da7d-d575-5574c1f939cc.png)


    


    Y = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 1, 1, 1, 1, 1]



    
![output_65_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/6ad8ad42-50f7-b22d-3c31-6c37d2ab8b76.png)


    


    Y = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 1, 1, 1, 1]



    
![output_65_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9ce6d154-dc14-b7ca-04b8-8d65805c0b90.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 1, 1, 1]



    
![output_65_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/95008090-3318-cb75-63a8-66d25b390eaf.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 1, 1]



    
![output_65_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/50a38924-640a-8843-40af-62155326ff9a.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 1]



    
![output_65_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/1a2ffd41-eb00-1d83-def0-71d230de5659.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_65_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/eb260894-0cbb-6b08-8d75-12e6ebf5283f.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = [0 if i > j else 1 for j in range(n_people)]

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_67_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/589d9da6-d420-3435-b987-591f8438ac64.png)


    



```python
depict_matrix(score_record)
```


    
![output_68_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/04bdd982-90c3-a1b2-efba-04ec43cab2f1.png)


    


# シナリオ4-1：「感染者は絶対見逃さない！だが非感染者の5割を誤判定する」の巻

ああ、なんかもう「悪の組織（架空）」の設定が無理になってきたので、やめますね。悪の組織（架空）は解散しました。悪意がなくてもこういう検査法は存在しうるので、まあ性能が悪いのは残念だけど、悪意はないと思うので、その挙動を生暖かく見守ってあげてください。以下、説明は省略〜っ。

分類性能評価指標の冒険は、まだ始まったばかりだ！　（Qiitaの次回作にご期待ください）


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 1:
            Y_pred.append(y)
        elif j % 2 == 0:
            Y_pred.append(1)
        else:
            Y_pred.append(0)

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_70_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/72199ce3-fc1e-2097-b4d2-6d3b7f4951a9.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_70_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/41dd34bd-d1a7-b1af-279b-dbcde758fdd3.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_70_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/2e75ac33-ba26-5dd1-cd1e-3c60aa03ca64.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 0, 1, 0, 1, 0, 1, 0]



    
![output_70_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/cdb64a6b-0b3a-0f2d-1474-7164fa0c6bc8.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 0, 1, 0, 1, 0]



    
![output_70_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/02bef38e-b67e-b3d1-a0b5-020c99f5858c.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 0, 1, 0, 1, 0]



    
![output_70_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/4977083d-bf1d-338b-0380-0f755660214c.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 0, 1, 0]



    
![output_70_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c418c5ff-360f-d476-4458-22cb95f5e106.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 0, 1, 0]



    
![output_70_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/f5229902-d941-81bd-a2ae-d48c451ef39c.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]



    
![output_70_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/5e52bb8b-e80a-5e52-6d22-085e78f793d4.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]



    
![output_70_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/a7b89b8d-1c8b-2d2d-8f8d-e4cf356a9b46.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_70_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9530b617-d09b-ba81-0655-06e9373970f4.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 1:
            Y_pred.append(y)
        elif j % 2 == 0:
            Y_pred.append(1)
        else:
            Y_pred.append(0)

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_72_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/932c5ad2-2029-4657-7f7b-13ed5ead7e8c.png)


    



```python
depict_matrix(score_record)
```


    
![output_73_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/d3037175-fadf-8bbb-cce7-9e874d31af62.png)


    


# シナリオ4-2：「感染者は絶対見逃さない！だが非感染者の3割を誤判定する」の巻


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 1:
            Y_pred.append(y)
        elif j % 3 == 2:
            Y_pred.append(1)
        else:
            Y_pred.append(0)

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_75_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/94c785b7-9502-e48b-0c01-13df46b6a206.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_75_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/f03245fe-32b3-3d09-1bdc-838fb4c54e28.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_75_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/8fdb9204-d1ff-9218-b461-1d379d1b2dde.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_75_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c445e18b-2b4a-47af-c8b4-9319204ec810.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 0, 1, 0, 0, 1, 0]



    
![output_75_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/bbe44491-d541-0cd2-1bc9-c827bc54ee26.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 0, 0, 1, 0]



    
![output_75_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/e25aa2f3-23bf-ede6-c361-be2cd864add5.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 0, 0, 1, 0]



    
![output_75_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9251d680-076e-1975-95b8-b44ce7f1c4b1.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 0, 1, 0]



    
![output_75_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/30e45ced-8d53-589d-ac02-f9987ab0d23d.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]



    
![output_75_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/a54310db-7982-dcc2-35e2-237135165628.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]



    
![output_75_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/16c1ad1e-6968-d528-52b8-f133649e8dba.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]



    
![output_75_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/05fd25d1-42b4-0e39-cdb2-fe9af0a3d0d8.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 1:
            Y_pred.append(y)
        elif j % 3 == 2:
            Y_pred.append(1)
        else:
            Y_pred.append(0)

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_77_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/e095a550-5611-ac19-04d1-d3766683c66e.png)


    



```python
depict_matrix(score_record)
```


    
![output_78_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/b7495bbf-2e7a-cbff-06cb-1e63c83c5d3b.png)


    


# シナリオ5-1：「非感染者は絶対陽性判定しない！だが感染者の5割を誤判定する」の巻



```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            Y_pred.append(y)
        else:
            Y_pred.append(j % 2)

    print("Y =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_80_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/9008d6b3-5588-18a3-cb05-4cfe987f3cab.png)


    


    Y = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_80_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/6364e85a-b095-b19f-96fe-20c36b1e1f63.png)


    


    Y = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 1, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_80_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/61ccea1a-3ec5-0483-8216-40f239ac14ea.png)


    


    Y = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 1, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_80_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c1e869e8-4b81-15aa-7d8b-64a621b71368.png)


    


    Y = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 1, 0, 1, 0, 0, 0, 0, 0, 0]



    
![output_80_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/cf677fbb-2a2b-24de-c88a-bc18154c52d2.png)


    


    Y = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [0, 1, 0, 1, 0, 0, 0, 0, 0, 0]



    
![output_80_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/83581442-95cb-9aad-815e-ae5a65bc8e9b.png)


    


    Y = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [0, 1, 0, 1, 0, 1, 0, 0, 0, 0]



    
![output_80_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/53adb4d4-5fcf-7af4-92b8-a1ee2b6c393b.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [0, 1, 0, 1, 0, 1, 0, 0, 0, 0]



    
![output_80_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c84756df-8481-1754-4174-d2e904a1c218.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [0, 1, 0, 1, 0, 1, 0, 1, 0, 0]



    
![output_80_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ec195f65-b84b-8d07-ab0f-ae4a7d59efc2.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [0, 1, 0, 1, 0, 1, 0, 1, 0, 0]



    
![output_80_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/d2ec52a0-2f0c-2bb9-73d6-ff6a4aea5d16.png)


    


    Y = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [0, 1, 0, 1, 0, 1, 0, 1, 0, 1]



    
![output_80_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/7b502c8e-cbfc-372c-db29-57c866606201.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            Y_pred.append(y)
        else:
            Y_pred.append(j % 2)

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_82_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/7a8b5e50-861e-1513-b074-f8e7526b60bb.png)


    



```python
depict_matrix(score_record)
```


    
![output_83_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/87600ec7-0243-0233-3d09-2833b63fa0e3.png)


    


# シナリオ5-2：「非感染者は絶対陽性判定しない！だが感染者の3割を誤判定する」の巻



```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            Y_pred.append(y)
        elif j % 3 == 2:
            Y_pred.append(0)
        else:
            Y_pred.append(1)

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_85_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/b77a10a5-4e17-020e-7f3b-e1a3d16bd883.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_85_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/84b5bac0-5df7-e53c-c3af-932a793b8525.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_85_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/f4f53715-c877-62e3-8d9b-44b99db2111b.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]



    
![output_85_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/74fb4909-712b-e54a-f4a3-33ea0d03d3a5.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 0, 0, 0, 0, 0, 0]



    
![output_85_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ebb40346-a6e7-1d60-a301-d850a4ee2922.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 0, 0, 0, 0]



    
![output_85_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/afee4d3a-dd38-ef38-2d24-7e4ea6e6f031.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 0, 0, 0, 0]



    
![output_85_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/183cdf60-5630-994a-b069-63f6949fa5c8.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 0, 0, 0]



    
![output_85_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/5f2bfcac-2a6a-caea-15d7-3539b4587104.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 0]



    
![output_85_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/4808cca3-e976-4a15-1307-0a421ebde163.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 0]



    
![output_85_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/395d7bb0-ae0b-ed0b-69da-5d6a5b54a886.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 1]



    
![output_85_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/fdd76369-6a59-4bbc-2021-53f2f23d37cb.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            Y_pred.append(y)
        elif j % 3 == 2:
            Y_pred.append(0)
        else:
            Y_pred.append(1)

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_87_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/3144ec83-db6d-d1d2-db79-d5b4591f763b.png)


    



```python
depict_matrix(score_record)
```


    
![output_88_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/f3376342-189c-0a36-35fe-864b452bd413.png)


    


# シナリオ6-1：「感染者・非感染者関係なく３割を誤判定する」の巻


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            if j % 3 == 2:
                Y_pred.append(1)
            else:
                Y_pred.append(0)
        else:
            if j % 3 == 2:
                Y_pred.append(0)
            else:
                Y_pred.append(1)

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_90_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/523bceb2-5f78-a3ac-9ef9-d24aad489b7a.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_90_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c30989af-9fb5-7b3d-9b4e-58037cb6cad6.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_90_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ef20dbfa-4c70-7c03-8fa8-af3c319f0a8f.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 0, 0, 1, 0, 0, 1, 0]



    
![output_90_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/cbf5dfb8-0799-a2d3-f4cd-31091d40aa8d.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 0, 1, 0, 0, 1, 0]



    
![output_90_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ec17b916-81a8-4f2b-b572-0af523c3ded1.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 1, 0, 0, 1, 0]



    
![output_90_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/055caebc-57ec-3ed8-64bf-5be5c70ab9f9.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 0, 0, 1, 0]



    
![output_90_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/dccae2a6-8ab4-c4d9-07da-0f5776e9434f.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 0, 1, 0]



    
![output_90_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ba6fc269-3776-d013-f1a4-a91181cde1b0.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 1, 0]



    
![output_90_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/01b24c72-6cb5-795a-d314-cc552f8b415a.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 0]



    
![output_90_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c558b0ba-729f-8ace-ef7a-bfd002737606.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 1]



    
![output_90_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ac9ef2d3-9d76-72b6-dcd7-02a0d93e111d.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            if j % 3 == 2:
                Y_pred.append(1)
            else:
                Y_pred.append(0)
        else:
            if j % 3 == 2:
                Y_pred.append(0)
            else:
                Y_pred.append(1)

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_92_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/347563fc-de09-0dc2-6a97-a5938195f0de.png)


    



```python
depict_matrix(score_record)
```


    
![output_93_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c648e0e3-3a29-08be-f43e-16da474baba9.png)


    


# シナリオ6-2：「感染者の１割・非感染者の３割を誤判定する」の巻



```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            if j % 3 == 2:
                Y_pred.append(1)
            else:
                Y_pred.append(0)
        else:
            if j % 10 == 9:
                Y_pred.append(0)
            else:
                Y_pred.append(1)

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_95_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/e1f87114-b9fe-54b1-56c9-8b2dbc9d939f.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_95_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/1e4aa159-2656-4d51-8240-b07d3e5b6790.png)


    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_95_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/0b73eb6e-e4e8-4477-4ee6-1886161c3b8a.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 0, 0, 1, 0, 0, 1, 0]



    
![output_95_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/f0a5efc0-c345-483c-4c1d-337a2a29e658.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 0, 1, 0, 0, 1, 0]



    
![output_95_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/75f83558-b50c-0da1-0577-a14eea1a33e8.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 0, 0, 1, 0]



    
![output_95_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/99f379ee-f217-acb7-fbdb-805f31825659.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 0, 0, 1, 0]



    
![output_95_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ba59824c-65b3-e8b5-95cc-a78475d14c19.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 0, 1, 0]



    
![output_95_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/182217e2-a0dc-1fca-b940-442e1b2a5df1.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]



    
![output_95_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/21d0457a-d738-1025-5d28-f518df0ad818.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]



    
![output_95_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/fe54d725-725e-c60e-fc98-4e822f956fa6.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]



    
![output_95_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/36a7c28b-ff66-d1db-0e79-c4544058491a.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            if j % 3 == 2:
                Y_pred.append(1)
            else:
                Y_pred.append(0)
        else:
            if j % 10 == 9:
                Y_pred.append(0)
            else:
                Y_pred.append(1)

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_97_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/8990f5fc-b057-c334-5750-9a6684223fc0.png)


    



```python
depict_matrix(score_record)
```


    
![output_98_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/ed5c7cd6-ddc3-3778-dda7-3cf6ef094405.png)


    


# シナリオ6-3：「感染者の３割・非感染者の１割を誤判定する」の巻


```python
n_people = 10
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            if j % 10 == 9:
                Y_pred.append(1)
            else:
                Y_pred.append(0)
        else:
            if j % 3 == 2:
                Y_pred.append(0)
            else:
                Y_pred.append(1)

    print("Y      =", Y)
    print("Y_pred =", Y_pred)
    depict_classification_metrics(Y, Y_pred)
```

    Y      = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [0, 0, 0, 0, 0, 0, 0, 0, 0, 1]



    
![output_100_1.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/5ce77b94-cf00-9616-145c-55c469d2eeac.png)


    


    Y      = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 0, 0, 0, 0, 0, 0, 0, 0, 1]



    

![output_100_3.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/a0bc140d-b416-9b3a-1363-fa39dbcb6151.png)




    


    Y      = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 0, 0, 0, 0, 0, 0, 1]



    
![output_100_5.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/385d07db-b11a-49f5-bead-c19b866fb486.png)


    


    Y      = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 0, 0, 0, 0, 0, 0, 1]



    
![output_100_7.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/912c929a-5f21-2671-14c2-460aefeb6f60.png)


    


    Y      = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 0, 0, 0, 0, 0, 1]



    
![output_100_9.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/8b4f0f87-924a-964f-e61e-d579844bf52a.png)


    


    Y      = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 0, 0, 0, 1]



    
![output_100_11.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/79833016-9907-226b-e797-d8f22adfc4f1.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 0, 0, 0, 1]



    
![output_100_13.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/fddec354-f8df-460e-5b64-6ec5acc5e588.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 0, 0, 1]



    
![output_100_15.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/95a5403b-2952-5d99-0c05-dc822c2510dd.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 1]



    
![output_100_17.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/5b2d88e3-72f0-2b4b-964f-816187a90b24.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 0]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 1]



    
![output_100_19.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/c5696560-290e-b089-cb24-78ce3e4fa513.png)


    


    Y      = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Y_pred = [1, 1, 0, 1, 1, 0, 1, 1, 0, 1]



    
![output_100_21.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/a6485efc-793f-cf10-c0e2-0213a4b32aab.png)


    



```python
n_people = 100
score_record = {}
for i in range(n_people + 1):
    Y = [1 if i > j else 0 for j in range(n_people)]
    Y_pred = []
    for j, y in enumerate(Y):
        if y == 0:
            if j % 10 == 9:
                Y_pred.append(1)
            else:
                Y_pred.append(0)
        else:
            if j % 3 == 2:
                Y_pred.append(0)
            else:
                Y_pred.append(1)

    if "p_infected" not in score_record.keys():
        score_record["p_infected"] = []
    score_record["p_infected"].append(i / n_people)

    for measure_name, measure in measures:
        if measure_name not in score_record.keys():
            score_record[measure_name] = []
        score_record[measure_name].append(measure(Y, Y_pred))
```


```python
depict_layer(score_record)
```


    
![output_102_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/55219ad3-97b1-82ea-1f4a-56f79831f1dc.png)


    



```python
depict_matrix(score_record)
```


    
![output_103_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/211162/06349012-5f19-c322-204c-5cecac7a05ba.png)